-- Logs begin at Wed 2022-08-03 04:12:52 UTC, end at Wed 2022-08-03 05:33:10 UTC. -- Aug 03 04:53:02.086770 dv-cinder-851279-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 03 04:53:02.094178 dv-cinder-851279-4-iscsi devstack@keystone.service[80342]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 03 04:53:02.098867 dv-cinder-851279-4-iscsi devstack@keystone.service[80342]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 3 04:53:02 2022] *** Aug 03 04:53:02.098867 dv-cinder-851279-4-iscsi devstack@keystone.service[80342]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 03 04:53:02.098867 dv-cinder-851279-4-iscsi devstack@keystone.service[80342]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 03 04:53:02.098867 dv-cinder-851279-4-iscsi devstack@keystone.service[80342]: nodename: dv-cinder-851279-4-iscsi Aug 03 04:53:02.098867 dv-cinder-851279-4-iscsi devstack@keystone.service[80342]: machine: x86_64 Aug 03 04:53:02.098867 dv-cinder-851279-4-iscsi devstack@keystone.service[80342]: clock source: unix Aug 03 04:53:02.098867 dv-cinder-851279-4-iscsi devstack@keystone.service[80342]: pcre jit disabled Aug 03 04:53:02.098867 dv-cinder-851279-4-iscsi devstack@keystone.service[80342]: detected number of CPU cores: 4 Aug 03 04:53:02.098867 dv-cinder-851279-4-iscsi devstack@keystone.service[80342]: current working directory: / Aug 03 04:53:02.098867 dv-cinder-851279-4-iscsi devstack@keystone.service[80342]: detected binary path: /usr/bin/uwsgi-core Aug 03 04:53:02.098867 dv-cinder-851279-4-iscsi devstack@keystone.service[80342]: your processes number limit is 47680 Aug 03 04:53:02.098867 dv-cinder-851279-4-iscsi devstack@keystone.service[80342]: your memory page size is 4096 bytes Aug 03 04:53:02.098867 dv-cinder-851279-4-iscsi devstack@keystone.service[80342]: detected max file descriptor number: 2048 Aug 03 04:53:02.098867 dv-cinder-851279-4-iscsi devstack@keystone.service[80342]: lock engine: pthread robust mutexes Aug 03 04:53:02.098867 dv-cinder-851279-4-iscsi devstack@keystone.service[80342]: thunder lock: enabled Aug 03 04:53:02.098867 dv-cinder-851279-4-iscsi devstack@keystone.service[80342]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 03 04:53:02.098867 dv-cinder-851279-4-iscsi devstack@keystone.service[80342]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 03 04:53:02.137102 dv-cinder-851279-4-iscsi devstack@keystone.service[80342]: Python main interpreter initialized at 0x55b96b1e69c0 Aug 03 04:53:02.137102 dv-cinder-851279-4-iscsi devstack@keystone.service[80342]: python threads support enabled Aug 03 04:53:02.137102 dv-cinder-851279-4-iscsi devstack@keystone.service[80342]: your server socket listen backlog is limited to 100 connections Aug 03 04:53:02.137102 dv-cinder-851279-4-iscsi devstack@keystone.service[80342]: your mercy for graceful operations on workers is 90 seconds Aug 03 04:53:02.137484 dv-cinder-851279-4-iscsi devstack@keystone.service[80342]: mapped 403077 bytes (393 KB) for 2 cores Aug 03 04:53:02.137562 dv-cinder-851279-4-iscsi devstack@keystone.service[80342]: *** Operational MODE: preforking *** Aug 03 04:53:02.137809 dv-cinder-851279-4-iscsi devstack@keystone.service[80342]: *** uWSGI is running in multiple interpreter mode *** Aug 03 04:53:02.137809 dv-cinder-851279-4-iscsi devstack@keystone.service[80342]: spawned uWSGI master process (pid: 80342) Aug 03 04:53:02.138017 dv-cinder-851279-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 03 04:53:02.142763 dv-cinder-851279-4-iscsi devstack@keystone.service[80342]: spawned uWSGI worker 1 (pid: 80355, cores: 1) Aug 03 04:53:02.142763 dv-cinder-851279-4-iscsi devstack@keystone.service[80342]: spawned uWSGI worker 2 (pid: 80356, cores: 1) Aug 03 04:53:02.142763 dv-cinder-851279-4-iscsi devstack@keystone.service[80342]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 03 04:53:07.733156 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 03 04:53:07.733156 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: from cryptography.utils import int_from_bytes Aug 03 04:53:07.733935 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 03 04:53:07.733935 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: from cryptography.utils import int_from_bytes Aug 03 04:53:07.734058 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 03 04:53:07.734058 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: from cryptography.utils import int_from_bytes Aug 03 04:53:07.734058 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 03 04:53:07.734058 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: from cryptography.utils import int_from_bytes Aug 03 04:53:07.785459 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 03 04:53:07.785651 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 03 04:53:07.785751 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 03 04:53:07.785844 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 03 04:53:07.785927 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] command line args: [] {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 03 04:53:07.786008 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] command line args: [] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 03 04:53:07.786086 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 03 04:53:07.786220 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 03 04:53:07.786303 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ================================================================================ {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 03 04:53:07.786408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ================================================================================ {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 03 04:53:07.786518 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] admin_token = **** {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.786671 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] admin_token = **** {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.786757 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] config_dir = [] {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.786951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] config_dir = [] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.787042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.787142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] config_source = [] {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.787230 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.787324 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] debug = True {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.787410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] config_source = [] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.787488 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] debug = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.787617 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.787839 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.787964 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.788102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.788266 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.788408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.788408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.788594 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.788742 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.788860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.788966 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] list_limit = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.789062 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] list_limit = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.789062 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] log_config_append = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.789275 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] log_config_append = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.789386 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.789492 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.789492 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] log_dir = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.789682 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] log_dir = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.789784 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] log_file = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.789886 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] log_file = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.789988 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.790094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.790278 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.790377 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.790377 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.790567 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.790567 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.790702 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.790777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.790933 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.791017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.791138 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.791225 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.791303 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.791422 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.791503 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.791575 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.791652 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.791728 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.791804 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.791920 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.792001 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.792157 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.792290 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.792410 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.792497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.792659 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.792743 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.792937 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.793031 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.793449 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.793537 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.793669 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] publish_errors = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.793746 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] publish_errors = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.793849 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.793934 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.794081 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.794209 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.794288 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.794359 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.794492 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.794592 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.794728 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.794808 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.794994 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] standard_threads = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.795075 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] standard_threads = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.795227 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.795306 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.795467 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.795553 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.795631 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.795708 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.795805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] use_journal = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.795889 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] use_journal = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.795971 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] use_json = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.796047 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] use_json = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.796124 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] use_stderr = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.796200 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] use_stderr = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.796302 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] use_syslog = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.796380 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] use_syslog = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.796533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.796613 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 04:53:07.796790 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.796876 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.797028 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.797106 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.797287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.797372 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.797526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.797604 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.797775 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.797860 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.798014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.798094 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.798261 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.798339 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.798497 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.798584 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.798745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.798906 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.799019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.799114 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.799268 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.799357 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.799504 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.799582 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.799751 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.799829 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.799987 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.800078 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.800233 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.800369 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.800490 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.801125 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.801239 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.801328 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.801495 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.801574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.801749 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.801829 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.801995 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.802081 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.802230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.802315 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.802482 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.802561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.802744 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.802834 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.803021 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.803102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.803267 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.803347 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.803499 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.803657 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.803732 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.803802 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.803954 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.804033 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.804191 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.804280 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.804438 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.804516 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.804685 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.804764 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.804915 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.805004 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.805147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.805231 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.805393 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.805477 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.805637 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.805725 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.805876 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.805954 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.806124 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.806202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.806356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.806441 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.806592 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.806671 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.806828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.806956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.807066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.807166 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.807302 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.807414 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.807533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.807675 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.807787 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.807906 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.807982 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.808092 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.808226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.808313 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.808519 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.808610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.808713 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.808843 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.808968 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.809084 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.809204 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.809307 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.809425 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.809516 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.809680 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.809760 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.809912 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.809993 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.810156 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.810239 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.810397 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.810485 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.810636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.810724 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.810897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.810981 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.811138 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.811216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.811371 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.811463 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.811621 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.811727 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.811859 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.811937 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.812098 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.812184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.812330 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.812410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.812591 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.812683 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.812830 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.812908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.813062 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.813142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.813302 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.813386 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.813539 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.813619 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.813810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.813889 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.814048 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.814131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.814298 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.814377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.814544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.814624 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.814781 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.814914 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.815044 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.815124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.815320 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.815400 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.815517 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.815597 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.815758 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.815837 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.815998 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.816079 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.816306 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.816911 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.817057 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.817194 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.817290 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.817436 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.817538 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.817677 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.817783 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.817919 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.818026 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.818158 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.818294 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.818418 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.818543 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.818663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.818790 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.818932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.819061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.819174 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.819311 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.819417 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.819556 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.819643 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.819802 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.819882 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.820053 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.820133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.820291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.820379 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.820529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.820610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.820768 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.820854 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.821003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.821090 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.821236 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.821315 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.821478 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.821558 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.821714 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.821801 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.821965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.822045 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.822205 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.822284 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.822461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.822548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.822722 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.822802 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.822988 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.823070 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.823222 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] auth.external = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.823316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] auth.external = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.823454 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.823540 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.823752 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.823865 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.823990 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.824071 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.824220 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] auth.password = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.824307 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] auth.password = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.824452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] auth.token = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.824532 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] auth.token = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.824694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.824774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.824931 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.825022 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.825181 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.825260 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.825412 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.825492 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.825662 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.825751 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.825913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.825993 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.826164 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.826253 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.826389 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] credential.caching = True {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.826476 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] credential.caching = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.826621 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.826708 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.826890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.826974 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.827142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.827224 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.827382 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.827463 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.827628 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.827708 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.827865 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.827947 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.828133 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.828220 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.828384 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.828464 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.828624 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.828711 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.828877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.828958 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.829163 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.829247 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.829412 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.829494 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.829681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.829767 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.829926 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.830011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.830198 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.830330 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.830415 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] federation.caching = True {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.830491 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] federation.caching = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.830646 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.830727 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.830918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.831002 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.831169 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.831250 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.831403 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.831483 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.831651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.831733 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.831914 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.831996 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.832176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.832256 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.832415 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.832497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.832671 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.832751 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.832910 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.832989 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.833162 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.833242 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.833398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] identity.caching = True {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.833478 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity.caching = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.833647 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.833729 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.833900 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.833981 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.834138 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.834217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.834377 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.834458 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.834626 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.834708 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.834887 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.834973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.835122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.835214 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.835374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.835456 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.835629 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.835709 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.835863 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.835944 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.836098 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.836179 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.836331 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.836412 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.836577 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.836666 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.836825 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.836905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.837077 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.837159 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.837326 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.837415 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.837576 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.837655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.837845 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.837924 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.838087 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.838175 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.838327 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.838409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.838560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.838641 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.838799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.838915 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.839055 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.839136 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.839293 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.839373 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.839537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.839628 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.839784 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.839866 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.840045 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.840125 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.840284 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.840371 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.840532 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.840619 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.840781 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.840868 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.841019 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.841108 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.841268 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.841348 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.841514 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.841594 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.841746 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.841835 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.841986 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.842075 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.842219 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.842298 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.842457 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.842544 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.842696 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.842777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.842961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.843044 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.843202 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.843294 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.843442 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.843523 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.843691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.843771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.843943 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.844039 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.844172 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.844252 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.844403 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.844483 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.844652 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.844740 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.844902 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.844982 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.845139 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.845220 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.845377 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.845468 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.845614 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.845695 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.845847 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.user = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.845927 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.846087 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.846176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.846346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.846426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.846579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.846659 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.846826 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.846977 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.847096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.847177 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.847345 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.847442 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.847607 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.847702 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.847841 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.847922 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.848079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.848177 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.848315 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.848398 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.848554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.848635 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.848785 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.848865 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.849033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.849115 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.849279 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.849363 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.849525 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.849606 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.849882 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.849965 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.850133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.850214 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.850367 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.850447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.850617 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.850699 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.850876 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.850960 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.851123 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.851254 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.851381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.851461 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.851590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.851671 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.851838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.851918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.852086 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.852168 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.852326 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.852407 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.852582 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.852662 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.852814 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.852896 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.853058 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.853138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.853301 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.853381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.853529 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.853605 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.853764 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.853845 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.854014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.854095 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.854256 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.854338 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.854488 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.854568 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.854728 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.854809 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.854995 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] resource.caching = True {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.855078 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] resource.caching = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.855255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.855335 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.855510 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.855581 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.855787 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.855885 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.856012 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.856094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.856270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.856351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.856507 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.856589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.856755 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.856835 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.856995 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.857076 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.857234 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.857316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.857473 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] role.caching = True {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.857560 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] role.caching = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.857720 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] role.driver = sql {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.857801 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] role.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.857952 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.858035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.858202 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.858283 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.858448 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.858529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.858694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.858777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.858966 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.859049 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.859213 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.859294 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.859459 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.859541 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.859703 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.859784 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.859949 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.860029 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.860179 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.860261 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.860423 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.860503 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.860668 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.860749 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.860913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.860995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.861158 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.861245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.861563 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.861652 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.861799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.861880 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.862043 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.862124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.862290 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.862378 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.862539 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.862625 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.862787 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.862953 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.863552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.863638 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.863818 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.863900 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.864072 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.864152 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.864318 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.864406 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.864550 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.864630 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.864783 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.864864 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.865015 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.865104 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.865268 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.865348 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.865520 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.865600 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.865769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.865858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.866005 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.866086 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.866241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.866322 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.866471 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.866560 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.866706 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] token.caching = True {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.866786 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] token.caching = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.866967 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.867049 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.867216 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.867307 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.867452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.867532 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.867705 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.867786 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.867953 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.868035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.868189 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.868269 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.868435 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.868515 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.868682 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.868763 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.868927 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.869007 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.869166 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.869246 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.869406 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.869488 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.869642 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.869722 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.869889 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.869969 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.870137 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.870219 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.870369 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.870449 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.870613 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.870693 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.870996 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.871083 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.871263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] database.connection = **** {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.871356 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.connection = **** {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.871512 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.871602 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.871759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.871839 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.871998 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.872087 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.872242 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.872323 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.872478 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.872558 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.872717 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.872805 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.872954 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.873035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.873191 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.873271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.873437 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.873594 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.873672 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.873745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.873909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.873990 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.874146 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.874239 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.874399 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.874480 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.874635 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.874714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.874904 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.874998 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.875140 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.875220 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.875383 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.875463 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.875619 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.875707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 04:53:07.875781 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 03 04:53:07.875853 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80356) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 03 04:53:07.905853 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:07.906005 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80356) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:07.906085 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:07.906177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80356) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:07.906286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:07.906374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80356) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:07.914758 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80356) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:07.914875 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:07.945777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:07.945934 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80356) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:07.946022 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:07.946097 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80356) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:07.998301 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80356) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:07.998789 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.048960 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80356) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.049188 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80356) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.049917 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.050155 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.114804 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80356) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.115052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80356) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.115194 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.115283 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80356) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.115430 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.115535 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80356) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.115658 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.115743 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80356) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.115880 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80356) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.115964 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.116097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.116314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.138875 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80356) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.139102 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80356) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.139194 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.139327 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80356) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.139426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.139544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80356) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.139628 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.139750 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80356) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.139832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.140028 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80356) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.140113 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.140243 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80356) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.140318 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.140456 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.151319 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80356) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.151706 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.164838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80356) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.165019 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80356) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.165108 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.165243 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80356) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.165340 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.165460 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80356) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.165554 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.165672 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80356) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.165757 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.165882 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80356) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.165967 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.166092 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80356) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.166174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.166375 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 04:53:08.183455 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.183661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 03 04:53:08.183996 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.184083 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.184196 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 03 04:53:08.184282 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 03 04:53:08.184466 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.184590 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.184738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.184826 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 03 04:53:08.184972 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.185078 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.185220 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.185341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.185603 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.185861 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.194946 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.195218 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.195483 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.195732 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.195818 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.196012 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.196285 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.196568 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.202594 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80356) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.203562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.207385 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80356) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.207663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.207942 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.208216 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.208325 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.208493 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.208605 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.208749 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.208887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.209015 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.209166 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.209283 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.209448 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.209531 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.209708 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.209791 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.209976 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.210058 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.210248 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.210518 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.210790 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.211084 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.233342 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.234615 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.234809 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80356) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.235110 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.236128 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.236410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.240495 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80356) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.240770 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.241048 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.241579 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.241862 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.242145 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.247624 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80356) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.247905 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.248816 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.249100 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.252121 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80356) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.252408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.252697 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.252987 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.253262 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.253545 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.253840 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.254008 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.254139 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.254300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.254596 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.254907 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.255201 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.255477 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.255755 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.256033 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.270133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.270403 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.272234 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.272518 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.274318 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80356) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.276588 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.277461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.277752 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.280026 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.280324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.282361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80356) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.284731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.286058 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80356) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.288450 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.289222 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.289702 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.289880 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.290173 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.290456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.290742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.291747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.292049 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.292336 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.292634 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.292914 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.293210 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.307448 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80356) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.307730 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.307990 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.308247 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.309940 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.310227 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.310498 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.310765 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.315163 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.316734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.317005 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.317725 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.319279 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.319557 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.319819 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80356) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.320096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.320382 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.320666 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.320948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.322378 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.322662 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.322988 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.323277 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.323573 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.332343 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80356) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.332618 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.332894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.333165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.333452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.333704 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.333984 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.335076 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.335359 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.335644 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.335926 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.336205 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.336479 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.336755 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.350068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80356) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.352869 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.353026 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80356) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.355886 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.356045 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.358529 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.358866 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.360105 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80356) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.360385 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.360663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.361363 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.362932 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.363776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.364062 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.367701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.369277 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80356) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.370528 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.372160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.372260 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.372533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.372816 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.373089 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.375128 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.375417 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.375701 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.376082 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.382076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80356) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.382361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.382640 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.384291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.384582 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.384862 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.389481 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80356) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.389756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.390039 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.390321 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.390592 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.390884 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.391196 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.391535 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.391753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 04:53:08.391904 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.392033 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.392201 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.392329 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.392490 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.392603 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.392752 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.392880 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.393017 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.393110 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.393274 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80356) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.393361 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.393613 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.394002 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.394322 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.394800 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.395107 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.395397 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.395662 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 04:53:08.427937 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b96b1e69c0 pid: 80356 (default app) Aug 03 04:53:08.430420 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b96b1e69c0 pid: 80355 (default app) Aug 03 04:53:08.431221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ace07-0e00-42cc-a94e-84ad87845a59 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:08.431511 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ace07-0e00-42cc-a94e-84ad87845a59 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:08.431786 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ace07-0e00-42cc-a94e-84ad87845a59 None None] PATH_INFO: `/v3` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:08.436342 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1/1] 10.222.0.40 () {50 vars in 791 bytes} [Wed Aug 3 04:53:08 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:16.741757 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f4bc0-8634-47f1-a854-787ee3ac07d7 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:16.742236 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f4bc0-8634-47f1-a854-787ee3ac07d7 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:16.742320 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f4bc0-8634-47f1-a854-787ee3ac07d7 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:16.746778 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1/2] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:53:16.752210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01b265-7e14-4800-93fe-be146869972a None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:16.752620 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01b265-7e14-4800-93fe-be146869972a None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:16.752776 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01b265-7e14-4800-93fe-be146869972a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:16.823740 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: INFO dbcounter [None req-8e01b265-7e14-4800-93fe-be146869972a None None] Registered counter for database keystone Aug 03 04:53:16.843156 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG oslo_db.sqlalchemy.engines [None req-8e01b265-7e14-4800-93fe-be146869972a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80356) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 03 04:53:16.850085 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writer thread running {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 03 04:53:17.031066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG passlib.handlers.bcrypt [None req-8e01b265-7e14-4800-93fe-be146869972a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80356) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 03 04:53:17.031491 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG passlib.handlers.bcrypt [None req-8e01b265-7e14-4800-93fe-be146869972a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80356) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 03 04:53:17.086137 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-8e01b265-7e14-4800-93fe-be146869972a None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:17.087778 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-8e01b265-7e14-4800-93fe-be146869972a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:17.218229 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 2/3] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 03 04:53:17.338742 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f4bc0-8634-47f1-a854-787ee3ac07d7 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:17.408497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: INFO dbcounter [None req-6a0f4bc0-8634-47f1-a854-787ee3ac07d7 None None] Registered counter for database keystone Aug 03 04:53:17.430462 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG oslo_db.sqlalchemy.engines [None req-6a0f4bc0-8634-47f1-a854-787ee3ac07d7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80355) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 03 04:53:17.436772 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writer thread running {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 03 04:53:17.555495 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f4bc0-8634-47f1-a854-787ee3ac07d7 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:17.559803 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb52279-e6d7-4a47-b5ad-e50c34072921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:17.560435 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb52279-e6d7-4a47-b5ad-e50c34072921 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:17.560721 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb52279-e6d7-4a47-b5ad-e50c34072921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:17.560998 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb52279-e6d7-4a47-b5ad-e50c34072921 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:17.569210 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb52279-e6d7-4a47-b5ad-e50c34072921 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:17.621044 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6eb52279-e6d7-4a47-b5ad-e50c34072921 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 03 04:53:17.621335 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb52279-e6d7-4a47-b5ad-e50c34072921 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:17.630790 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 2/4] 10.222.0.40 () {58 vars in 1324 bytes} [Wed Aug 3 04:53:17 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 03 04:53:17.636948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01b265-7e14-4800-93fe-be146869972a None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:17.651873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01b265-7e14-4800-93fe-be146869972a None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:17.656558 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea90b697-ebbe-4979-b660-1b2bcc182754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:17.657679 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90b697-ebbe-4979-b660-1b2bcc182754 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:17.657961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90b697-ebbe-4979-b660-1b2bcc182754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:17.658230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90b697-ebbe-4979-b660-1b2bcc182754 admin admin] PATH_INFO: `/v3/projects/6917683e025f412781fd852422d27ce7` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:17.659816 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea90b697-ebbe-4979-b660-1b2bcc182754 admin admin] RBAC: Authorizing `identity:get_project(project_id=6917683e025f412781fd852422d27ce7)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:17.710648 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ea90b697-ebbe-4979-b660-1b2bcc182754 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 03 04:53:17.711018 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea90b697-ebbe-4979-b660-1b2bcc182754 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:17.715716 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 3/5] 10.222.0.40 () {56 vars in 1207 bytes} [Wed Aug 3 04:53:17 2022] GET /identity/v3/projects/6917683e025f412781fd852422d27ce7 => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:19.675128 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f460a-2b23-4ce4-b9b0-2f91ff933935 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:19.675566 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f460a-2b23-4ce4-b9b0-2f91ff933935 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:19.675656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f460a-2b23-4ce4-b9b0-2f91ff933935 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:19.680337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 3/6] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:19.685785 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a068c-9bce-4298-b51f-dd682c15ec10 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:19.686187 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a068c-9bce-4298-b51f-dd682c15ec10 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:19.686352 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a068c-9bce-4298-b51f-dd682c15ec10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:19.706737 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-c45a068c-9bce-4298-b51f-dd682c15ec10 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:19.708325 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-c45a068c-9bce-4298-b51f-dd682c15ec10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:19.716275 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 4/7] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:19 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 03 04:53:19.737865 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f460a-2b23-4ce4-b9b0-2f91ff933935 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:19.750170 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f460a-2b23-4ce4-b9b0-2f91ff933935 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:19.754420 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5752293-eafd-4ad0-acc9-2ed4535413c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:19.755551 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5752293-eafd-4ad0-acc9-2ed4535413c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:19.755858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5752293-eafd-4ad0-acc9-2ed4535413c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:19.756138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5752293-eafd-4ad0-acc9-2ed4535413c6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:19.763776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5752293-eafd-4ad0-acc9-2ed4535413c6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:19.765551 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5752293-eafd-4ad0-acc9-2ed4535413c6 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:19.778450 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 4/8] 10.222.0.40 () {58 vars in 1324 bytes} [Wed Aug 3 04:53:19 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 04:53:19.784057 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a068c-9bce-4298-b51f-dd682c15ec10 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:19.793230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a068c-9bce-4298-b51f-dd682c15ec10 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:19.797619 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d8999-8394-4ae1-9287-b91be97d2bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:19.798255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d8999-8394-4ae1-9287-b91be97d2bed admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:19.798540 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d8999-8394-4ae1-9287-b91be97d2bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:19.798816 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d8999-8394-4ae1-9287-b91be97d2bed admin admin] PATH_INFO: `/v3/users/d53b9368b2734205a2bdcd0bc2796161` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:19.800214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d8999-8394-4ae1-9287-b91be97d2bed admin admin] RBAC: Authorizing `identity:get_user(user_id=d53b9368b2734205a2bdcd0bc2796161)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:19.802563 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d8999-8394-4ae1-9287-b91be97d2bed admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:19.804136 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 5/9] 10.222.0.40 () {56 vars in 1198 bytes} [Wed Aug 3 04:53:19 2022] GET /identity/v3/users/d53b9368b2734205a2bdcd0bc2796161 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:21.884800 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae72b3a-1ac5-4c94-90e6-52558417b308 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:21.885392 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae72b3a-1ac5-4c94-90e6-52558417b308 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:21.885392 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae72b3a-1ac5-4c94-90e6-52558417b308 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:21.886710 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 5/10] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:21.891897 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6deda-f644-4fce-947e-6022c154709f None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:21.892300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6deda-f644-4fce-947e-6022c154709f None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:21.892585 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6deda-f644-4fce-947e-6022c154709f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:21.912537 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-18e6deda-f644-4fce-947e-6022c154709f None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:21.913789 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-18e6deda-f644-4fce-947e-6022c154709f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:21.935264 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81495671-84e1-4cc4-b5bd-e742d9f5f45f None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:21.935555 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81495671-84e1-4cc4-b5bd-e742d9f5f45f None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:21.935838 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81495671-84e1-4cc4-b5bd-e742d9f5f45f None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:21.937162 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 6/11] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:53:21.942751 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c906067-f3eb-4b17-9591-b5d9c2f263cb None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:21.943062 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c906067-f3eb-4b17-9591-b5d9c2f263cb None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:21.943339 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c906067-f3eb-4b17-9591-b5d9c2f263cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:21.984353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG passlib.handlers.bcrypt [None req-9c906067-f3eb-4b17-9591-b5d9c2f263cb None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80355) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 03 04:53:21.984768 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG passlib.handlers.bcrypt [None req-9c906067-f3eb-4b17-9591-b5d9c2f263cb None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80355) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 03 04:53:21.994644 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 6/12] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:53:22.017970 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6deda-f644-4fce-947e-6022c154709f None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:22.031026 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-9c906067-f3eb-4b17-9591-b5d9c2f263cb None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:22.032617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9c906067-f3eb-4b17-9591-b5d9c2f263cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:22.040854 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:22.123406 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6deda-f644-4fce-947e-6022c154709f None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:22.134918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 7/13] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 04:53:22.156962 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c906067-f3eb-4b17-9591-b5d9c2f263cb None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:22.157276 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d023f-5163-43b9-ada8-9dc4da4a5a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:22.157927 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d023f-5163-43b9-ada8-9dc4da4a5a14 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:22.158213 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d023f-5163-43b9-ada8-9dc4da4a5a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:22.158494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d023f-5163-43b9-ada8-9dc4da4a5a14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:22.171422 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d023f-5163-43b9-ada8-9dc4da4a5a14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:53:22.180398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-827d023f-5163-43b9-ada8-9dc4da4a5a14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:53:22.180398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:53:22.180398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:53:22.180398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:53:22.180398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:53:22.180398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:53:22.180398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:53:22.180398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:53:22.180398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:53:22.180398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:53:22.180398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:53:22.180398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:53:22.180398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:53:22.180398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:53:22.180398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:53:22.180398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:53:22.181524 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:53:22.181524 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:53:22.181524 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:53:22.181524 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:53:22.181524 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:53:22.181524 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:53:22.181524 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:53:22.181524 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:53:22.181524 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:53:22.181524 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:53:22.181524 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:53:22.181524 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:53:22.181524 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:53:22.181524 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:53:22.181524 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:53:22.181524 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:53:22.182595 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:53:22.182595 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:53:22.182595 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:53:22.182595 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:53:22.182595 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:53:22.182595 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:53:22.182595 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:53:22.182595 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:53:22.182595 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:53:22.182595 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:53:22.182595 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:53:22.182595 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:53:22.182595 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 7/14] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:53:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:53:22.187437 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d023f-5163-43b9-ada8-9dc4da4a5a14 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:22.262680 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c906067-f3eb-4b17-9591-b5d9c2f263cb None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:22.265737 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:22.268720 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:22.289052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d023f-5163-43b9-ada8-9dc4da4a5a14 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:22.303471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d412f0-dd68-487c-8a3f-1d19c95d2ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:22.304335 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d412f0-dd68-487c-8a3f-1d19c95d2ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:22.304619 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d412f0-dd68-487c-8a3f-1d19c95d2ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:22.304901 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d412f0-dd68-487c-8a3f-1d19c95d2ec5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:22.323517 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d3e87a-7093-4542-8c9c-d3fad3821c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:22.324131 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d3e87a-7093-4542-8c9c-d3fad3821c6a None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:22.324411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d3e87a-7093-4542-8c9c-d3fad3821c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:22.324686 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d3e87a-7093-4542-8c9c-d3fad3821c6a None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:22.325400 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d3e87a-7093-4542-8c9c-d3fad3821c6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:22.325707 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d3e87a-7093-4542-8c9c-d3fad3821c6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:22.327133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d3e87a-7093-4542-8c9c-d3fad3821c6a None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:22.338993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 8/15] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:53:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:53:22.344561 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d3e87a-7093-4542-8c9c-d3fad3821c6a None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:22.372242 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d412f0-dd68-487c-8a3f-1d19c95d2ec5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:22.373774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d412f0-dd68-487c-8a3f-1d19c95d2ec5 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:22.448725 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d3e87a-7093-4542-8c9c-d3fad3821c6a None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:22.471896 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 8/16] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:53:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 03 04:53:22.478011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d412f0-dd68-487c-8a3f-1d19c95d2ec5 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:22.481087 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:22.484150 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c411385-2adf-4bf8-a91c-0fd58d71af15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:22.484793 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c411385-2adf-4bf8-a91c-0fd58d71af15 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:22.485076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c411385-2adf-4bf8-a91c-0fd58d71af15 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:22.485353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c411385-2adf-4bf8-a91c-0fd58d71af15 None admin] PATH_INFO: `/v3/users/d53b9368b2734205a2bdcd0bc2796161` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:22.486561 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c411385-2adf-4bf8-a91c-0fd58d71af15 None admin] RBAC: Authorizing `identity:get_user(user_id=d53b9368b2734205a2bdcd0bc2796161)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:22.488241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c411385-2adf-4bf8-a91c-0fd58d71af15 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:22.490375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 9/17] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:53:22 2022] GET /identity/v3/users/d53b9368b2734205a2bdcd0bc2796161 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:53:22.495712 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c411385-2adf-4bf8-a91c-0fd58d71af15 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:22.509955 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:22.580835 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d412f0-dd68-487c-8a3f-1d19c95d2ec5 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:22.619855 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e05870-afd2-45ea-b0c4-81299946f15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:22.620747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e05870-afd2-45ea-b0c4-81299946f15c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:22.621035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e05870-afd2-45ea-b0c4-81299946f15c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:22.621324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e05870-afd2-45ea-b0c4-81299946f15c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:22.643115 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e05870-afd2-45ea-b0c4-81299946f15c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:22.644623 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e05870-afd2-45ea-b0c4-81299946f15c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:22.663814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-20e05870-afd2-45ea-b0c4-81299946f15c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:53:22.663814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:53:22.663814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:53:22.663814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:53:22.663814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:53:22.663814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:53:22.663814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:53:22.663814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:53:22.663814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:53:22.663814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:53:22.663814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:53:22.663814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:53:22.663814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:53:22.663814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:53:22.663814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:53:22.664696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:53:22.664696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:53:22.664696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:53:22.664696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:53:22.664696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:53:22.664696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:53:22.664696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:53:22.664696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:53:22.664696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:53:22.664696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:53:22.664696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:53:22.664696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:53:22.664696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:53:22.664696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:53:22.664696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:53:22.664696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:53:22.664696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:53:22.665981 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:53:22.665981 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:53:22.665981 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:53:22.665981 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:53:22.665981 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 9/18] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:53:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:53:22.670790 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e05870-afd2-45ea-b0c4-81299946f15c None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:22.705897 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c411385-2adf-4bf8-a91c-0fd58d71af15 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:22.726834 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:22.739453 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd1b10b-90fc-47f2-a4c4-2a46c1263d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:22.740107 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1b10b-90fc-47f2-a4c4-2a46c1263d05 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:22.740392 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1b10b-90fc-47f2-a4c4-2a46c1263d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:22.740678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1b10b-90fc-47f2-a4c4-2a46c1263d05 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:22.741855 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd1b10b-90fc-47f2-a4c4-2a46c1263d05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:22.743374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd1b10b-90fc-47f2-a4c4-2a46c1263d05 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:22.744947 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 10/19] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:53:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:53:22.750548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd1b10b-90fc-47f2-a4c4-2a46c1263d05 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:22.771457 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e05870-afd2-45ea-b0c4-81299946f15c None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:22.805265 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220975d-b029-499f-8a13-fd77f013fb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:22.805831 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220975d-b029-499f-8a13-fd77f013fb67 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:22.806114 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220975d-b029-499f-8a13-fd77f013fb67 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:22.806481 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220975d-b029-499f-8a13-fd77f013fb67 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:22.807100 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5220975d-b029-499f-8a13-fd77f013fb67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:22.807406 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5220975d-b029-499f-8a13-fd77f013fb67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:22.808816 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5220975d-b029-499f-8a13-fd77f013fb67 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:22.826191 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 10/20] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:53:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:53:22.834031 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:22.855269 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd1b10b-90fc-47f2-a4c4-2a46c1263d05 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:22.892189 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d40b09-b8ff-4f68-bdfe-10066a98d451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:22.892911 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d40b09-b8ff-4f68-bdfe-10066a98d451 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:22.893147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d40b09-b8ff-4f68-bdfe-10066a98d451 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:22.893424 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d40b09-b8ff-4f68-bdfe-10066a98d451 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:22.894237 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d40b09-b8ff-4f68-bdfe-10066a98d451 None admin] RBAC: Adding query filter params (role.id=e39d4ecea4c64d5794e1079c9be3ad36, scope.domain.id=default, user.id=d53b9368b2734205a2bdcd0bc2796161) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:22.894549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d40b09-b8ff-4f68-bdfe-10066a98d451 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:22.897687 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d40b09-b8ff-4f68-bdfe-10066a98d451 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:22.908131 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 11/21] 10.222.0.40 () {56 vars in 1322 bytes} [Wed Aug 3 04:53:22 2022] GET /identity/v3/role_assignments?user.id=d53b9368b2734205a2bdcd0bc2796161&scope.domain.id=default&role.id=e39d4ecea4c64d5794e1079c9be3ad36 => generated 233 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:53:25.245924 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04befe9-4316-411b-9cc4-04cb23c9c596 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:25.246537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04befe9-4316-411b-9cc4-04cb23c9c596 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:25.246537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04befe9-4316-411b-9cc4-04cb23c9c596 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:25.247879 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 11/22] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:25.249371 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe783d-d0c8-4120-b854-88113cbf7ccb None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:25.249750 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe783d-d0c8-4120-b854-88113cbf7ccb None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:25.249935 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe783d-d0c8-4120-b854-88113cbf7ccb None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:25.251305 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 12/23] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:25.254080 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44854638-7a20-43cd-99eb-5c6a2718126e None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:25.254901 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44854638-7a20-43cd-99eb-5c6a2718126e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:25.255189 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44854638-7a20-43cd-99eb-5c6a2718126e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:25.259569 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea52fbc-ad8d-42f5-ba9e-c7b0d38eb2d1 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:25.259861 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea52fbc-ad8d-42f5-ba9e-c7b0d38eb2d1 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:25.260143 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea52fbc-ad8d-42f5-ba9e-c7b0d38eb2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:25.291286 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-2ea52fbc-ad8d-42f5-ba9e-c7b0d38eb2d1 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:25.291964 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-44854638-7a20-43cd-99eb-5c6a2718126e None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:25.292517 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-2ea52fbc-ad8d-42f5-ba9e-c7b0d38eb2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:25.293157 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-44854638-7a20-43cd-99eb-5c6a2718126e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:25.400428 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 12/24] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:53:25.421257 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 13/25] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 04:53:25.425329 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44854638-7a20-43cd-99eb-5c6a2718126e None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:25.465379 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea52fbc-ad8d-42f5-ba9e-c7b0d38eb2d1 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:25.496367 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:25.570600 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44854638-7a20-43cd-99eb-5c6a2718126e None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:25.596799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:25.608042 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49672d74-efff-4d54-bf84-07dc5536bdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:25.609158 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49672d74-efff-4d54-bf84-07dc5536bdf4 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:25.609939 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49672d74-efff-4d54-bf84-07dc5536bdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:25.610502 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49672d74-efff-4d54-bf84-07dc5536bdf4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:25.626619 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49672d74-efff-4d54-bf84-07dc5536bdf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:53:25.627549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea52fbc-ad8d-42f5-ba9e-c7b0d38eb2d1 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:25.634330 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-49672d74-efff-4d54-bf84-07dc5536bdf4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:53:25.634330 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:53:25.634330 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:53:25.634330 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:53:25.634330 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:53:25.634330 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:53:25.634330 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:53:25.634330 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:53:25.634330 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:53:25.634330 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:53:25.634330 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:53:25.634330 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:53:25.634330 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:53:25.634330 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:53:25.634330 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:53:25.634330 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:53:25.635217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:53:25.635217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:53:25.635217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:53:25.635217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:53:25.635217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:53:25.635217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:53:25.635217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:53:25.635217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:53:25.635217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:53:25.635217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:53:25.635217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:53:25.635217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:53:25.635217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:53:25.635217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:53:25.635217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:53:25.635217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:53:25.635970 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:53:25.635970 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:53:25.635970 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:53:25.635970 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:53:25.635970 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:53:25.635970 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:53:25.635970 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:53:25.635970 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:53:25.635970 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:53:25.635970 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:53:25.635970 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:53:25.635970 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:53:25.637558 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 13/26] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:53:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:53:25.637558 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d30a5-4165-43db-9ddc-387c403392fd None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:25.637558 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d30a5-4165-43db-9ddc-387c403392fd None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:25.637895 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d30a5-4165-43db-9ddc-387c403392fd None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:25.641235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 14/27] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:25.642254 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e465c-bb9b-45cb-860d-deb978dc25c9 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:25.642518 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e465c-bb9b-45cb-860d-deb978dc25c9 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:25.642792 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e465c-bb9b-45cb-860d-deb978dc25c9 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:25.644021 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 15/28] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:25.644737 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e465c-bb9b-45cb-860d-deb978dc25c9 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:25.679392 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fdc193-cc99-43d9-ac60-123b5209b48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:25.680365 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdc193-cc99-43d9-ac60-123b5209b48e None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:25.680845 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdc193-cc99-43d9-ac60-123b5209b48e None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:25.681326 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdc193-cc99-43d9-ac60-123b5209b48e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:25.692903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fdc193-cc99-43d9-ac60-123b5209b48e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:25.694730 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fdc193-cc99-43d9-ac60-123b5209b48e None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:25.711500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-98fdc193-cc99-43d9-ac60-123b5209b48e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:53:25.711500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:53:25.711500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:53:25.711500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:53:25.711500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:53:25.711500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:53:25.711500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:53:25.711500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:53:25.711500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:53:25.711500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:53:25.711500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:53:25.711500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:53:25.711500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:53:25.711500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:53:25.711500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:53:25.712359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:53:25.712359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:53:25.712359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:53:25.712359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:53:25.712359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:53:25.712359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:53:25.712359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:53:25.712359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:53:25.712359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:53:25.712359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:53:25.712359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:53:25.712359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:53:25.712359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:53:25.712359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:53:25.712359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:53:25.712359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:53:25.712359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:53:25.713162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:53:25.713162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:53:25.713162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:53:25.713162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:53:25.715047 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 14/29] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:53:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:53:25.716572 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6811a-cca2-46de-9720-8f4bd392ec48 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:25.717077 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6811a-cca2-46de-9720-8f4bd392ec48 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:25.717552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6811a-cca2-46de-9720-8f4bd392ec48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:25.736621 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-e0b6811a-cca2-46de-9720-8f4bd392ec48 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:25.738123 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-e0b6811a-cca2-46de-9720-8f4bd392ec48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:25.760813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e465c-bb9b-45cb-860d-deb978dc25c9 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:25.818465 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 15/30] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:53:25.821355 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd5ad9-e467-401a-8469-55219c03b030 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:25.821866 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd5ad9-e467-401a-8469-55219c03b030 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:25.822341 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd5ad9-e467-401a-8469-55219c03b030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:25.844071 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-a5fd5ad9-e467-401a-8469-55219c03b030 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:25.845680 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-a5fd5ad9-e467-401a-8469-55219c03b030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:25.881258 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:25.906057 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:25.912552 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d308df-285d-4970-aedf-71695456c3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:25.914442 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d308df-285d-4970-aedf-71695456c3ef None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:25.914945 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d308df-285d-4970-aedf-71695456c3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:25.915413 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d308df-285d-4970-aedf-71695456c3ef None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:25.916455 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d308df-285d-4970-aedf-71695456c3ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:25.916997 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d308df-285d-4970-aedf-71695456c3ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:25.918603 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d308df-285d-4970-aedf-71695456c3ef None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:25.924973 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 16/31] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:53:25.925939 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd5ad9-e467-401a-8469-55219c03b030 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:25.932783 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 16/32] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:53:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:25.933911 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d308df-285d-4970-aedf-71695456c3ef None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:26.032085 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd5ad9-e467-401a-8469-55219c03b030 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:26.044706 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d308df-285d-4970-aedf-71695456c3ef None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:26.069580 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a36754-a6ba-4df8-b5c5-9df03cc2f2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:26.070258 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a36754-a6ba-4df8-b5c5-9df03cc2f2f3 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:26.070536 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a36754-a6ba-4df8-b5c5-9df03cc2f2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:26.070816 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a36754-a6ba-4df8-b5c5-9df03cc2f2f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:26.071747 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a36754-a6ba-4df8-b5c5-9df03cc2f2f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:26.071868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a36754-a6ba-4df8-b5c5-9df03cc2f2f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:26.073236 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a36754-a6ba-4df8-b5c5-9df03cc2f2f3 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:26.083877 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bdbad1-9658-4207-a133-0db3206a54b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:26.084767 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdbad1-9658-4207-a133-0db3206a54b7 None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:26.085278 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdbad1-9658-4207-a133-0db3206a54b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:26.085764 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdbad1-9658-4207-a133-0db3206a54b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:26.086685 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bdbad1-9658-4207-a133-0db3206a54b7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:26.088329 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bdbad1-9658-4207-a133-0db3206a54b7 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:26.092350 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 17/33] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:26.093056 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a36754-a6ba-4df8-b5c5-9df03cc2f2f3 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:26.143372 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:26.199853 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a36754-a6ba-4df8-b5c5-9df03cc2f2f3 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:26.238810 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be70ef7-e92e-4afd-9dac-468e9a61e7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:26.239547 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be70ef7-e92e-4afd-9dac-468e9a61e7ee None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:26.239837 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be70ef7-e92e-4afd-9dac-468e9a61e7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:26.240204 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be70ef7-e92e-4afd-9dac-468e9a61e7ee None admin] PATH_INFO: `/v3/users/d53b9368b2734205a2bdcd0bc2796161` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:26.241446 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be70ef7-e92e-4afd-9dac-468e9a61e7ee None admin] RBAC: Authorizing `identity:get_user(user_id=d53b9368b2734205a2bdcd0bc2796161)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:26.243020 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be70ef7-e92e-4afd-9dac-468e9a61e7ee None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:26.245155 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 18/34] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:53:26 2022] GET /identity/v3/users/d53b9368b2734205a2bdcd0bc2796161 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:26.246069 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be70ef7-e92e-4afd-9dac-468e9a61e7ee None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:26.341981 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 17/35] 10.222.0.40 () {60 vars in 1131 bytes} [Wed Aug 3 04:53:25 2022] POST /identity/v3/roles => generated 219 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:53:26.343666 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bdbad1-9658-4207-a133-0db3206a54b7 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:26.361997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be70ef7-e92e-4afd-9dac-468e9a61e7ee None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:26.384052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:26.387906 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:26.412600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e08f0-c98f-4cfb-9b29-d44a899a2348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:26.413978 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e08f0-c98f-4cfb-9b29-d44a899a2348 None admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:26.414411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e08f0-c98f-4cfb-9b29-d44a899a2348 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:26.414831 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e08f0-c98f-4cfb-9b29-d44a899a2348 None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:26.415917 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e08f0-c98f-4cfb-9b29-d44a899a2348 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:26.418223 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e08f0-c98f-4cfb-9b29-d44a899a2348 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:26.489235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bdbad1-9658-4207-a133-0db3206a54b7 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:26.529572 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb857fb-a59a-4b2a-862b-1079d57d08a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:26.530228 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb857fb-a59a-4b2a-862b-1079d57d08a2 None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:26.530515 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb857fb-a59a-4b2a-862b-1079d57d08a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:26.530791 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb857fb-a59a-4b2a-862b-1079d57d08a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:26.531788 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb857fb-a59a-4b2a-862b-1079d57d08a2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:26.533488 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb857fb-a59a-4b2a-862b-1079d57d08a2 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:26.662355 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 19/36] 10.222.0.40 () {60 vars in 1131 bytes} [Wed Aug 3 04:53:26 2022] POST /identity/v3/roles => generated 225 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:53:26.663354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e08f0-c98f-4cfb-9b29-d44a899a2348 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:26.701546 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 18/37] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Aug 3 04:53:26 2022] POST /identity/v3/projects => generated 301 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:53:26.749018 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b048deb-8c4f-4ecc-a383-c4e23a1aa1b9 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:26.749315 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b048deb-8c4f-4ecc-a383-c4e23a1aa1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:26.749588 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b048deb-8c4f-4ecc-a383-c4e23a1aa1b9 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:26.751144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 19/38] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:53:26.759221 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4e598-2833-4425-8ac6-c0acc0ac4491 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:26.760955 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4e598-2833-4425-8ac6-c0acc0ac4491 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:26.761463 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4e598-2833-4425-8ac6-c0acc0ac4491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:26.786674 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-2aa4e598-2833-4425-8ac6-c0acc0ac4491 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:26.789442 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2aa4e598-2833-4425-8ac6-c0acc0ac4491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:26.816526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e08f0-c98f-4cfb-9b29-d44a899a2348 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:26.881129 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2661ea89-9c0d-4299-8234-540328365bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:26.881929 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661ea89-9c0d-4299-8234-540328365bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:26.882217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661ea89-9c0d-4299-8234-540328365bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:26.882494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661ea89-9c0d-4299-8234-540328365bf7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:26.884262 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2661ea89-9c0d-4299-8234-540328365bf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:26.886234 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2661ea89-9c0d-4299-8234-540328365bf7 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:26.888031 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 20/39] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:53:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:26.894747 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7871344-ab10-4813-b912-1951494ec305 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:26.895155 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7871344-ab10-4813-b912-1951494ec305 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:26.895447 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7871344-ab10-4813-b912-1951494ec305 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:26.897248 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 21/40] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:26 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:26.901562 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7871344-ab10-4813-b912-1951494ec305 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:26.910886 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 20/41] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 04:53:26.914960 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef06515-b9b6-4958-8bfe-59deabd83138 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:26.914960 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef06515-b9b6-4958-8bfe-59deabd83138 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:26.914960 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef06515-b9b6-4958-8bfe-59deabd83138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:26.936590 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-1ef06515-b9b6-4958-8bfe-59deabd83138 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:26.937915 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1ef06515-b9b6-4958-8bfe-59deabd83138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:26.944897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:26.963515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:27.025208 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 21/42] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:53:27.026042 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef06515-b9b6-4958-8bfe-59deabd83138 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:27.038750 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7871344-ab10-4813-b912-1951494ec305 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:27.075488 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4dee8d-20f7-457e-9c6e-f4026b7c2a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:27.076177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dee8d-20f7-457e-9c6e-f4026b7c2a77 None admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:27.076474 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dee8d-20f7-457e-9c6e-f4026b7c2a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:27.076761 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dee8d-20f7-457e-9c6e-f4026b7c2a77 None admin] PATH_INFO: `/v3/domains/default/users/d53b9368b2734205a2bdcd0bc2796161/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:27.078911 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4dee8d-20f7-457e-9c6e-f4026b7c2a77 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d53b9368b2734205a2bdcd0bc2796161, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:27.081413 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4dee8d-20f7-457e-9c6e-f4026b7c2a77 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:27.136134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef06515-b9b6-4958-8bfe-59deabd83138 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:27.165633 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 22/43] 10.222.0.40 () {58 vars in 1361 bytes} [Wed Aug 3 04:53:26 2022] PUT /identity/v3/domains/default/users/d53b9368b2734205a2bdcd0bc2796161/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 04:53:27.166764 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4dee8d-20f7-457e-9c6e-f4026b7c2a77 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:27.170562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd44599-d333-4906-9cf6-12907d2cde6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:27.171241 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd44599-d333-4906-9cf6-12907d2cde6e None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:27.171530 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd44599-d333-4906-9cf6-12907d2cde6e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:27.171844 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd44599-d333-4906-9cf6-12907d2cde6e None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:27.172594 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd44599-d333-4906-9cf6-12907d2cde6e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:27.174062 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd44599-d333-4906-9cf6-12907d2cde6e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:27.248899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:27.256174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 22/44] 10.222.0.40 () {60 vars in 1131 bytes} [Wed Aug 3 04:53:27 2022] POST /identity/v3/roles => generated 223 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:53:27.293563 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4dee8d-20f7-457e-9c6e-f4026b7c2a77 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:27.295660 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:27.339706 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd72355-5604-4ca2-9fe9-e21a8c3e5e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:27.340394 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd72355-5604-4ca2-9fe9-e21a8c3e5e80 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:27.340687 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd72355-5604-4ca2-9fe9-e21a8c3e5e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:27.340972 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd72355-5604-4ca2-9fe9-e21a8c3e5e80 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:27.342359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd72355-5604-4ca2-9fe9-e21a8c3e5e80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:27.344465 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd72355-5604-4ca2-9fe9-e21a8c3e5e80 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:27.346656 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 23/45] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:53:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:27.353133 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd44599-d333-4906-9cf6-12907d2cde6e None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:27.471980 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd44599-d333-4906-9cf6-12907d2cde6e None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:27.506265 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f28b1-cbdf-4697-8b1d-699d984ca133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:27.506918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f28b1-cbdf-4697-8b1d-699d984ca133 None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:27.507211 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f28b1-cbdf-4697-8b1d-699d984ca133 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:27.507493 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f28b1-cbdf-4697-8b1d-699d984ca133 None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:27.508185 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f28b1-cbdf-4697-8b1d-699d984ca133 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:27.509646 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f28b1-cbdf-4697-8b1d-699d984ca133 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:27.601178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 23/46] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Aug 3 04:53:27 2022] POST /identity/v3/projects => generated 312 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 04:53:29.264208 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba496e-a27d-49b0-8931-18b0d295e92b None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:29.264754 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba496e-a27d-49b0-8931-18b0d295e92b None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:29.264754 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba496e-a27d-49b0-8931-18b0d295e92b None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:29.266118 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 24/47] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:29.271588 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d95457-1fdf-4742-a5ea-97beb1c0babe None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:29.271991 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d95457-1fdf-4742-a5ea-97beb1c0babe None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:29.272171 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d95457-1fdf-4742-a5ea-97beb1c0babe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:29.291462 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-e3d95457-1fdf-4742-a5ea-97beb1c0babe None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:29.292664 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e3d95457-1fdf-4742-a5ea-97beb1c0babe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:29.340417 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:29.360257 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 24/48] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:53:29.385251 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba496e-a27d-49b0-8931-18b0d295e92b None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:29.492237 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba496e-a27d-49b0-8931-18b0d295e92b None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:29.525682 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cab766-afa7-43b5-a418-0e24d734d370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:29.526257 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cab766-afa7-43b5-a418-0e24d734d370 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:29.526544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cab766-afa7-43b5-a418-0e24d734d370 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:29.526825 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cab766-afa7-43b5-a418-0e24d734d370 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:29.537675 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cab766-afa7-43b5-a418-0e24d734d370 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:53:29.542663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-f4cab766-afa7-43b5-a418-0e24d734d370 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:53:29.542663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:53:29.542663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:53:29.542663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:53:29.542663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:53:29.542663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:53:29.542663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:53:29.542663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:53:29.542663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:53:29.542663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:53:29.542663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:53:29.542663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:53:29.542663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:53:29.542663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:53:29.542663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:53:29.542663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:53:29.543543 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:53:29.543543 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:53:29.543543 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:53:29.543543 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:53:29.543543 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:53:29.543543 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:53:29.543543 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:53:29.543543 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:53:29.543543 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:53:29.543543 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:53:29.543543 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:53:29.543543 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:53:29.543543 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:53:29.543543 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:53:29.543543 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:53:29.543543 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:53:29.544317 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:53:29.544317 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:53:29.544317 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:53:29.544317 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:53:29.544317 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:53:29.544317 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:53:29.544317 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:53:29.544317 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:53:29.544317 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:53:29.544317 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:53:29.544317 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:53:29.544317 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:53:29.544317 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 25/49] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:53:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:53:29.551161 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d95457-1fdf-4742-a5ea-97beb1c0babe None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:29.636402 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0188a30-b5e5-4755-af78-65ffdb38762a None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:29.636693 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0188a30-b5e5-4755-af78-65ffdb38762a None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:29.636975 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0188a30-b5e5-4755-af78-65ffdb38762a None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:29.638329 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 26/50] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:53:29.643408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1b548-a0f3-459e-8b48-198164c3ec6d None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:29.643699 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1b548-a0f3-459e-8b48-198164c3ec6d None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:29.643977 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1b548-a0f3-459e-8b48-198164c3ec6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:29.649804 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d95457-1fdf-4742-a5ea-97beb1c0babe None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:29.663319 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-cfb1b548-a0f3-459e-8b48-198164c3ec6d None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:29.665136 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-cfb1b548-a0f3-459e-8b48-198164c3ec6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:29.684842 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e6256c-abe5-455a-9dc0-21c269498833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:29.685466 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6256c-abe5-455a-9dc0-21c269498833 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:29.685752 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6256c-abe5-455a-9dc0-21c269498833 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:29.686038 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6256c-abe5-455a-9dc0-21c269498833 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:29.686753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e6256c-abe5-455a-9dc0-21c269498833 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:29.687090 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e6256c-abe5-455a-9dc0-21c269498833 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:29.688470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e6256c-abe5-455a-9dc0-21c269498833 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:29.689797 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:29.697018 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 25/51] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:53:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:29.702774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e6256c-abe5-455a-9dc0-21c269498833 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:29.734373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 27/52] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 04:53:29.750599 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:29.756679 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb1b548-a0f3-459e-8b48-198164c3ec6d None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:29.805477 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e6256c-abe5-455a-9dc0-21c269498833 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:29.840466 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8fdc4c-0028-454a-9f2d-8ad532ca38ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:29.841116 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8fdc4c-0028-454a-9f2d-8ad532ca38ac None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:29.841414 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8fdc4c-0028-454a-9f2d-8ad532ca38ac None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:29.841696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8fdc4c-0028-454a-9f2d-8ad532ca38ac None admin] PATH_INFO: `/v3/users/d53b9368b2734205a2bdcd0bc2796161` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:29.843141 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8fdc4c-0028-454a-9f2d-8ad532ca38ac None admin] RBAC: Authorizing `identity:get_user(user_id=d53b9368b2734205a2bdcd0bc2796161)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:29.844605 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8fdc4c-0028-454a-9f2d-8ad532ca38ac None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:29.846213 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 26/53] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:53:29 2022] GET /identity/v3/users/d53b9368b2734205a2bdcd0bc2796161 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:29.851600 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8fdc4c-0028-454a-9f2d-8ad532ca38ac None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:29.857210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb1b548-a0f3-459e-8b48-198164c3ec6d None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:29.891777 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089afc9-17c1-411d-bc60-2545b5667022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:29.892786 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089afc9-17c1-411d-bc60-2545b5667022 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:29.893089 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089afc9-17c1-411d-bc60-2545b5667022 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:29.893374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089afc9-17c1-411d-bc60-2545b5667022 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:29.895497 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f089afc9-17c1-411d-bc60-2545b5667022 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:29.896965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f089afc9-17c1-411d-bc60-2545b5667022 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:29.898597 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 28/54] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:53:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:53:29.904367 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089afc9-17c1-411d-bc60-2545b5667022 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:29.931715 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:29.950934 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8fdc4c-0028-454a-9f2d-8ad532ca38ac None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:29.964092 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:29.984666 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d932385-4bdd-4b14-b81d-d5b1b735e71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:29.985302 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d932385-4bdd-4b14-b81d-d5b1b735e71a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:29.985590 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d932385-4bdd-4b14-b81d-d5b1b735e71a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:29.985865 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d932385-4bdd-4b14-b81d-d5b1b735e71a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:29.987061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d932385-4bdd-4b14-b81d-d5b1b735e71a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:29.988567 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d932385-4bdd-4b14-b81d-d5b1b735e71a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:29.990595 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 27/55] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:53:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:53:29.996173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d932385-4bdd-4b14-b81d-d5b1b735e71a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:30.005218 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089afc9-17c1-411d-bc60-2545b5667022 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:30.039927 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e648e-f1c4-4936-a1f7-a55ac22ccecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:30.040549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e648e-f1c4-4936-a1f7-a55ac22ccecf None admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:30.040832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e648e-f1c4-4936-a1f7-a55ac22ccecf None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:30.041108 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e648e-f1c4-4936-a1f7-a55ac22ccecf None admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:30.041788 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68e648e-f1c4-4936-a1f7-a55ac22ccecf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:30.043278 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68e648e-f1c4-4936-a1f7-a55ac22ccecf None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:30.095380 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d932385-4bdd-4b14-b81d-d5b1b735e71a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:30.128437 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09289ade-ad01-4700-98bf-efbf267fb146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:30.129588 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09289ade-ad01-4700-98bf-efbf267fb146 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:30.129881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09289ade-ad01-4700-98bf-efbf267fb146 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:30.130162 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09289ade-ad01-4700-98bf-efbf267fb146 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:30.130980 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09289ade-ad01-4700-98bf-efbf267fb146 None admin] RBAC: Adding query filter params (role.id=e39d4ecea4c64d5794e1079c9be3ad36, scope.domain.id=default, user.id=d53b9368b2734205a2bdcd0bc2796161) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:30.131296 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09289ade-ad01-4700-98bf-efbf267fb146 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:30.132755 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09289ade-ad01-4700-98bf-efbf267fb146 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:30.133225 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 29/56] 10.222.0.40 () {60 vars in 1140 bytes} [Wed Aug 3 04:53:29 2022] POST /identity/v3/projects => generated 298 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 04:53:30.142542 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 28/57] 10.222.0.40 () {56 vars in 1322 bytes} [Wed Aug 3 04:53:29 2022] GET /identity/v3/role_assignments?user.id=d53b9368b2734205a2bdcd0bc2796161&scope.domain.id=default&role.id=e39d4ecea4c64d5794e1079c9be3ad36 => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:53:32.216242 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32eebae-e616-45fb-8381-7aaca5893d20 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:32.216759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32eebae-e616-45fb-8381-7aaca5893d20 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:32.216845 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32eebae-e616-45fb-8381-7aaca5893d20 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:32.218196 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 30/58] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:32.223779 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e028dbf-25e5-44ee-927b-166d17a893ef None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:32.224169 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e028dbf-25e5-44ee-927b-166d17a893ef None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:32.224352 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e028dbf-25e5-44ee-927b-166d17a893ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:32.242684 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-7e028dbf-25e5-44ee-927b-166d17a893ef None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:32.243905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7e028dbf-25e5-44ee-927b-166d17a893ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:32.286143 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:32.319526 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 29/59] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:53:32.343797 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32eebae-e616-45fb-8381-7aaca5893d20 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:32.424236 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:32.445367 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32eebae-e616-45fb-8381-7aaca5893d20 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:32.478888 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e0b05-d1c1-4a0d-b700-85032e6f6e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:32.479481 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e0b05-d1c1-4a0d-b700-85032e6f6e32 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:32.479768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e0b05-d1c1-4a0d-b700-85032e6f6e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:32.480052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e0b05-d1c1-4a0d-b700-85032e6f6e32 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:32.481203 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e0b05-d1c1-4a0d-b700-85032e6f6e32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:32.482673 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e0b05-d1c1-4a0d-b700-85032e6f6e32 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:32.484299 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 31/60] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:53:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:32.490508 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e028dbf-25e5-44ee-927b-166d17a893ef None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:32.588303 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e028dbf-25e5-44ee-927b-166d17a893ef None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:32.621020 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42c2cb-58a8-468c-8bbf-f74f9c384fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:32.621634 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42c2cb-58a8-468c-8bbf-f74f9c384fde None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:32.621916 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42c2cb-58a8-468c-8bbf-f74f9c384fde None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:32.622188 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42c2cb-58a8-468c-8bbf-f74f9c384fde None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:32.622893 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42c2cb-58a8-468c-8bbf-f74f9c384fde None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:32.624336 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42c2cb-58a8-468c-8bbf-f74f9c384fde None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:32.737086 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 30/61] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 3 04:53:32 2022] POST /identity/v3/users => generated 275 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:53:35.749901 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9d101-3279-44e7-aac7-6d95f469aa82 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:35.750909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9d101-3279-44e7-aac7-6d95f469aa82 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:35.751395 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9d101-3279-44e7-aac7-6d95f469aa82 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:35.753008 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 32/62] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:35.758588 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638145ae-2c37-4a74-bff8-55eb638f15bb None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:35.759449 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638145ae-2c37-4a74-bff8-55eb638f15bb None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:35.759931 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638145ae-2c37-4a74-bff8-55eb638f15bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:35.783115 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-638145ae-2c37-4a74-bff8-55eb638f15bb None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:35.785294 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-638145ae-2c37-4a74-bff8-55eb638f15bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:35.825410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:35.893650 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 31/63] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:53:35.930052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f9d101-3279-44e7-aac7-6d95f469aa82 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:36.079525 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f9d101-3279-44e7-aac7-6d95f469aa82 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:36.115404 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237ecb1-2fe7-4900-a1af-38db011d000d None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:36.116067 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237ecb1-2fe7-4900-a1af-38db011d000d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:36.116553 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237ecb1-2fe7-4900-a1af-38db011d000d None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:36.118131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 32/64] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:53:36.123422 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4a3fa-168a-4f1b-a03a-28ba2d7f83fe None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:36.124135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4a3fa-168a-4f1b-a03a-28ba2d7f83fe None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:36.124635 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4a3fa-168a-4f1b-a03a-28ba2d7f83fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:36.139717 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e326e5-fcb1-4ba9-8bfb-f38700a83a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:36.140712 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e326e5-fcb1-4ba9-8bfb-f38700a83a14 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:36.141730 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e326e5-fcb1-4ba9-8bfb-f38700a83a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:36.142253 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e326e5-fcb1-4ba9-8bfb-f38700a83a14 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:36.147671 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-daf4a3fa-168a-4f1b-a03a-28ba2d7f83fe None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:36.149142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-daf4a3fa-168a-4f1b-a03a-28ba2d7f83fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:36.158323 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e326e5-fcb1-4ba9-8bfb-f38700a83a14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:53:36.165047 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-c9e326e5-fcb1-4ba9-8bfb-f38700a83a14 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:53:36.165047 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:53:36.165047 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:53:36.165047 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:53:36.165047 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:53:36.165047 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:53:36.165047 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:53:36.165047 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:53:36.165047 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:53:36.165047 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:53:36.165047 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:53:36.165047 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:53:36.165047 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:53:36.165047 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:53:36.165047 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:53:36.165047 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:53:36.165918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:53:36.165918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:53:36.165918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:53:36.165918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:53:36.165918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:53:36.165918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:53:36.165918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:53:36.165918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:53:36.165918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:53:36.165918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:53:36.165918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:53:36.165918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:53:36.165918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:53:36.165918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:53:36.165918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:53:36.165918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:53:36.167814 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:53:36.167814 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:53:36.167814 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:53:36.167814 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:53:36.167814 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:53:36.167814 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:53:36.167814 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:53:36.167814 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:53:36.167814 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:53:36.167814 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:53:36.167814 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:53:36.167814 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:53:36.167814 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 33/65] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:53:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:53:36.169307 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a12d33-f6ab-4ed5-a7c6-62223548af5a None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:36.169594 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a12d33-f6ab-4ed5-a7c6-62223548af5a None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:36.169886 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a12d33-f6ab-4ed5-a7c6-62223548af5a None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:36.172133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 34/66] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:36.177610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a12d33-f6ab-4ed5-a7c6-62223548af5a None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:36.207340 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:36.245917 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 33/67] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:53:36.247593 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc85b32-eda9-46e3-bb4b-54ad65e1d634 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:36.248143 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc85b32-eda9-46e3-bb4b-54ad65e1d634 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:36.248639 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc85b32-eda9-46e3-bb4b-54ad65e1d634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:36.282247 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-6bc85b32-eda9-46e3-bb4b-54ad65e1d634 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:36.283845 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6bc85b32-eda9-46e3-bb4b-54ad65e1d634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:36.333889 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a12d33-f6ab-4ed5-a7c6-62223548af5a None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:36.359038 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 34/68] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:53:36.360250 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc85b32-eda9-46e3-bb4b-54ad65e1d634 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:36.397913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d1959-916d-4dd4-8429-aed6466780d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:36.398137 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:36.399320 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d1959-916d-4dd4-8429-aed6466780d8 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:36.399782 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d1959-916d-4dd4-8429-aed6466780d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:36.400230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d1959-916d-4dd4-8429-aed6466780d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:36.401514 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3d1959-916d-4dd4-8429-aed6466780d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:36.402273 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3d1959-916d-4dd4-8429-aed6466780d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:36.403903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3d1959-916d-4dd4-8429-aed6466780d8 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:36.412975 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 35/69] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 3 04:53:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:36.414239 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d1959-916d-4dd4-8429-aed6466780d8 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:36.488300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc85b32-eda9-46e3-bb4b-54ad65e1d634 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:36.497391 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:36.534219 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d1959-916d-4dd4-8429-aed6466780d8 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:36.545058 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88fb011-ec15-4f58-941a-367df4def53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:36.545709 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fb011-ec15-4f58-941a-367df4def53c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:36.545994 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fb011-ec15-4f58-941a-367df4def53c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:36.546281 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fb011-ec15-4f58-941a-367df4def53c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:36.558002 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88fb011-ec15-4f58-941a-367df4def53c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:53:36.562968 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-e88fb011-ec15-4f58-941a-367df4def53c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 04:53:36.562968 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:53:36.562968 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:53:36.562968 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:53:36.562968 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:53:36.562968 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:53:36.562968 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:53:36.562968 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:53:36.562968 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:53:36.562968 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:53:36.562968 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:53:36.562968 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:53:36.562968 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:53:36.562968 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:53:36.562968 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:53:36.562968 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:53:36.564030 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:53:36.564030 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:53:36.564030 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:53:36.564030 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:53:36.564030 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:53:36.564030 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:53:36.564030 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:53:36.564030 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:53:36.564030 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:53:36.564030 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:53:36.564030 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:53:36.564030 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:53:36.564030 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:53:36.564030 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:53:36.564030 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:53:36.564030 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:53:36.570419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:53:36.570419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:53:36.570419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:53:36.570419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:53:36.570419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:53:36.570419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:53:36.570419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:53:36.570419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:53:36.570419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:53:36.570419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:53:36.570419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 04:53:36.570419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:53:36.570419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 35/70] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 3 04:53:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:53:36.570419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88fb011-ec15-4f58-941a-367df4def53c None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:36.579793 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32802078-341c-4efa-86e9-b37f0fb07d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:36.580442 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32802078-341c-4efa-86e9-b37f0fb07d71 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:36.580737 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32802078-341c-4efa-86e9-b37f0fb07d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:36.581051 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32802078-341c-4efa-86e9-b37f0fb07d71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:36.591242 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32802078-341c-4efa-86e9-b37f0fb07d71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:53:36.596592 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-32802078-341c-4efa-86e9-b37f0fb07d71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:53:36.596592 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:53:36.596592 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:53:36.596592 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:53:36.596592 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:53:36.596592 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:53:36.596592 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:53:36.596592 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:53:36.596592 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:53:36.596592 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:53:36.596592 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:53:36.596592 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:53:36.596592 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:53:36.596592 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:53:36.596592 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:53:36.596592 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:53:36.598133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:53:36.598133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:53:36.598133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:53:36.598133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:53:36.598133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:53:36.598133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:53:36.598133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:53:36.598133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:53:36.598133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:53:36.598133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:53:36.598133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:53:36.598133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:53:36.598133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:53:36.598133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:53:36.598133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:53:36.598133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:53:36.599129 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:53:36.599129 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:53:36.599129 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:53:36.599129 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:53:36.599129 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:53:36.599129 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:53:36.599129 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:53:36.599129 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:53:36.599129 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:53:36.599129 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:53:36.599129 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:53:36.599129 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:53:36.599129 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 36/71] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:53:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:53:36.599913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c712cc-328b-405a-adef-3385ac3a9742 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:36.599913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c712cc-328b-405a-adef-3385ac3a9742 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:36.600053 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c712cc-328b-405a-adef-3385ac3a9742 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:36.601188 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 37/72] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:36.606266 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5169709-f0e4-4bb7-ae8b-f21d12cc1be7 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:36.606562 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5169709-f0e4-4bb7-ae8b-f21d12cc1be7 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:36.606866 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5169709-f0e4-4bb7-ae8b-f21d12cc1be7 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:36.608115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 38/73] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:36.608819 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5169709-f0e4-4bb7-ae8b-f21d12cc1be7 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:36.650506 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:36.672223 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88fb011-ec15-4f58-941a-367df4def53c None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:36.706706 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c36ded-47c8-4b90-a3b0-2fa3b1b9cd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:36.707377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c36ded-47c8-4b90-a3b0-2fa3b1b9cd78 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:36.707657 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c36ded-47c8-4b90-a3b0-2fa3b1b9cd78 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:36.707930 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c36ded-47c8-4b90-a3b0-2fa3b1b9cd78 None admin] PATH_INFO: `/v3/users/b2cf0210955141068b832fc376bdc5ab` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:36.714582 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5169709-f0e4-4bb7-ae8b-f21d12cc1be7 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:36.723226 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c36ded-47c8-4b90-a3b0-2fa3b1b9cd78 None admin] RBAC: Authorizing `identity:get_user(user_id=b2cf0210955141068b832fc376bdc5ab)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:36.724936 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c36ded-47c8-4b90-a3b0-2fa3b1b9cd78 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:36.726700 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 36/74] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:53:36 2022] GET /identity/v3/users/b2cf0210955141068b832fc376bdc5ab => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:36.727495 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c36ded-47c8-4b90-a3b0-2fa3b1b9cd78 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:36.747964 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:36.753867 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e5fcfd-e86a-4da0-90b2-fe6e148a0dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:36.754504 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5fcfd-e86a-4da0-90b2-fe6e148a0dae None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:36.754786 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5fcfd-e86a-4da0-90b2-fe6e148a0dae None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:36.755099 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5fcfd-e86a-4da0-90b2-fe6e148a0dae None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:36.755818 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e5fcfd-e86a-4da0-90b2-fe6e148a0dae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:36.756132 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e5fcfd-e86a-4da0-90b2-fe6e148a0dae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:36.757520 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e5fcfd-e86a-4da0-90b2-fe6e148a0dae None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:36.766308 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 39/75] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:53:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:36.767715 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c390342-e56b-467b-8ce3-8b679d74ddad None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:36.767951 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c390342-e56b-467b-8ce3-8b679d74ddad None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:36.768235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c390342-e56b-467b-8ce3-8b679d74ddad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:36.786505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-6c390342-e56b-467b-8ce3-8b679d74ddad None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:36.787756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-6c390342-e56b-467b-8ce3-8b679d74ddad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:36.829593 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c36ded-47c8-4b90-a3b0-2fa3b1b9cd78 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:36.856098 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 40/76] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:53:36.871008 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdae83e-8670-45f6-98a5-2c56c686af18 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:36.871008 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdae83e-8670-45f6-98a5-2c56c686af18 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:36.871008 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdae83e-8670-45f6-98a5-2c56c686af18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:36.875514 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-5fdae83e-8670-45f6-98a5-2c56c686af18 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:36.876728 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-5fdae83e-8670-45f6-98a5-2c56c686af18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:36.880415 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b0489-2ce7-4595-842b-0894131aa941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:36.881049 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b0489-2ce7-4595-842b-0894131aa941 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:36.881354 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b0489-2ce7-4595-842b-0894131aa941 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:36.881623 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b0489-2ce7-4595-842b-0894131aa941 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:36.882348 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b0489-2ce7-4595-842b-0894131aa941 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:36.882688 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b0489-2ce7-4595-842b-0894131aa941 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:36.884109 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b0489-2ce7-4595-842b-0894131aa941 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:36.889948 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:36.893336 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 37/77] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:53:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:36.895772 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b0489-2ce7-4595-842b-0894131aa941 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:36.945872 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 41/78] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:53:36.947027 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdae83e-8670-45f6-98a5-2c56c686af18 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:37.001083 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:37.011654 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b0489-2ce7-4595-842b-0894131aa941 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:37.045881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f1ae2-f695-471c-96b5-9df5daa0777b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:37.046336 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f1ae2-f695-471c-96b5-9df5daa0777b None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:37.046625 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f1ae2-f695-471c-96b5-9df5daa0777b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:37.046948 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f1ae2-f695-471c-96b5-9df5daa0777b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:37.048679 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdae83e-8670-45f6-98a5-2c56c686af18 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:37.082112 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50f0f4-9900-4988-9f06-8ea6d098b156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:37.082879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50f0f4-9900-4988-9f06-8ea6d098b156 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:37.083176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50f0f4-9900-4988-9f06-8ea6d098b156 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:37.083461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50f0f4-9900-4988-9f06-8ea6d098b156 None admin] PATH_INFO: `/v3/users/b2cf0210955141068b832fc376bdc5ab` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:37.084639 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e50f0f4-9900-4988-9f06-8ea6d098b156 None admin] RBAC: Authorizing `identity:get_user(user_id=b2cf0210955141068b832fc376bdc5ab)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:37.086111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e50f0f4-9900-4988-9f06-8ea6d098b156 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:37.087929 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 42/79] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:53:36 2022] GET /identity/v3/users/b2cf0210955141068b832fc376bdc5ab => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:37.088515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50f0f4-9900-4988-9f06-8ea6d098b156 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:37.117070 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:37.119444 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617f1ae2-f695-471c-96b5-9df5daa0777b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:37.120920 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617f1ae2-f695-471c-96b5-9df5daa0777b None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:37.199846 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50f0f4-9900-4988-9f06-8ea6d098b156 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:37.220208 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:37.228255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 38/80] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:53:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:53:37.228255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f1ae2-f695-471c-96b5-9df5daa0777b None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:37.234654 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e571f7-ccdd-4ea1-88a8-03a3a9076a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:37.235328 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e571f7-ccdd-4ea1-88a8-03a3a9076a3a None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:37.235620 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e571f7-ccdd-4ea1-88a8-03a3a9076a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:37.235902 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e571f7-ccdd-4ea1-88a8-03a3a9076a3a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:37.237070 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e571f7-ccdd-4ea1-88a8-03a3a9076a3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:37.238566 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e571f7-ccdd-4ea1-88a8-03a3a9076a3a None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:37.240290 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 43/81] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:53:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:37.240933 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e571f7-ccdd-4ea1-88a8-03a3a9076a3a None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:37.326510 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:37.329441 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f1ae2-f695-471c-96b5-9df5daa0777b None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:37.340240 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e571f7-ccdd-4ea1-88a8-03a3a9076a3a None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:37.363931 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9e163-b2d8-45b6-b3af-97c537e378b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:37.364574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9e163-b2d8-45b6-b3af-97c537e378b1 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:37.364866 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9e163-b2d8-45b6-b3af-97c537e378b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:37.365141 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9e163-b2d8-45b6-b3af-97c537e378b1 None admin] PATH_INFO: `/v3/users/d53b9368b2734205a2bdcd0bc2796161` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:37.366297 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f9e163-b2d8-45b6-b3af-97c537e378b1 None admin] RBAC: Authorizing `identity:get_user(user_id=d53b9368b2734205a2bdcd0bc2796161)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:37.367804 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f9e163-b2d8-45b6-b3af-97c537e378b1 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:37.369406 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 39/82] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:53:37 2022] GET /identity/v3/users/d53b9368b2734205a2bdcd0bc2796161 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:37.374176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9e163-b2d8-45b6-b3af-97c537e378b1 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:37.374310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f8f00-7b3e-4de9-9e67-060170487089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:37.374310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f8f00-7b3e-4de9-9e67-060170487089 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:37.374310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f8f00-7b3e-4de9-9e67-060170487089 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:37.374560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f8f00-7b3e-4de9-9e67-060170487089 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:37.385019 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f8f00-7b3e-4de9-9e67-060170487089 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:53:37.390306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-d47f8f00-7b3e-4de9-9e67-060170487089 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:53:37.390306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:53:37.390306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:53:37.390306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:53:37.390306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:53:37.390306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:53:37.390306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:53:37.390306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:53:37.390306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:53:37.390306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:53:37.390306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:53:37.390306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:53:37.390306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:53:37.390306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:53:37.390306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:53:37.390306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:53:37.391219 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:53:37.391219 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:53:37.391219 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:53:37.391219 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:53:37.391219 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:53:37.391219 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:53:37.391219 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:53:37.391219 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:53:37.391219 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:53:37.391219 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:53:37.391219 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:53:37.391219 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:53:37.391219 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:53:37.391219 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:53:37.391219 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:53:37.391219 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:53:37.391966 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:53:37.391966 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:53:37.391966 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:53:37.391966 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:53:37.391966 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:53:37.391966 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:53:37.391966 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:53:37.391966 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:53:37.391966 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:53:37.391966 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:53:37.391966 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:53:37.391966 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:53:37.391966 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 44/83] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:53:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:53:37.392630 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f8f00-7b3e-4de9-9e67-060170487089 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:37.439685 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:37.474691 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9e163-b2d8-45b6-b3af-97c537e378b1 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:37.491239 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f8f00-7b3e-4de9-9e67-060170487089 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:37.508672 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c152794-3871-4e50-8112-e5719a733ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:37.509289 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c152794-3871-4e50-8112-e5719a733ffb None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:37.509575 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c152794-3871-4e50-8112-e5719a733ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:37.509855 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c152794-3871-4e50-8112-e5719a733ffb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:37.524930 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac8c5e-2ce3-4150-b908-84570ae6608e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:37.525570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac8c5e-2ce3-4150-b908-84570ae6608e None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:37.525852 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac8c5e-2ce3-4150-b908-84570ae6608e None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:37.526130 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac8c5e-2ce3-4150-b908-84570ae6608e None admin] PATH_INFO: `/v3/projects/1952481417bf4a2cad0b08d5a7676b61` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:37.527569 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac8c5e-2ce3-4150-b908-84570ae6608e None admin] RBAC: Authorizing `identity:get_project(project_id=1952481417bf4a2cad0b08d5a7676b61)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:37.529038 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac8c5e-2ce3-4150-b908-84570ae6608e None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:37.530690 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 45/84] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:53:37 2022] GET /identity/v3/projects/1952481417bf4a2cad0b08d5a7676b61 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:37.531467 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac8c5e-2ce3-4150-b908-84570ae6608e None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:37.550916 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:37.584525 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c152794-3871-4e50-8112-e5719a733ffb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:37.585920 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c152794-3871-4e50-8112-e5719a733ffb None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:37.631232 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac8c5e-2ce3-4150-b908-84570ae6608e None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:37.648346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:37.667084 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9cf73-9fdb-42aa-aab5-d2a930117558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:37.667723 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9cf73-9fdb-42aa-aab5-d2a930117558 None admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:37.668006 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9cf73-9fdb-42aa-aab5-d2a930117558 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:37.668283 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9cf73-9fdb-42aa-aab5-d2a930117558 None admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:37.669082 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a9cf73-9fdb-42aa-aab5-d2a930117558 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:37.670554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a9cf73-9fdb-42aa-aab5-d2a930117558 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:37.685755 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 40/85] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:53:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:53:37.687833 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c152794-3871-4e50-8112-e5719a733ffb None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:37.751985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:37.771217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 46/86] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 3 04:53:37 2022] POST /identity/v3/users => generated 289 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:53:37.771913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9cf73-9fdb-42aa-aab5-d2a930117558 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:37.799637 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c152794-3871-4e50-8112-e5719a733ffb None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:37.843837 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad033dd8-c0e9-4bd7-b73c-70e1c589ba76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:37.844457 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad033dd8-c0e9-4bd7-b73c-70e1c589ba76 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:37.844719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad033dd8-c0e9-4bd7-b73c-70e1c589ba76 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:37.845039 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad033dd8-c0e9-4bd7-b73c-70e1c589ba76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:37.887637 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9cf73-9fdb-42aa-aab5-d2a930117558 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:37.922394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad033dd8-c0e9-4bd7-b73c-70e1c589ba76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:37.923918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad033dd8-c0e9-4bd7-b73c-70e1c589ba76 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:37.925760 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c807c-b1cd-47e8-a1a4-424f751179f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:37.926601 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c807c-b1cd-47e8-a1a4-424f751179f1 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:37.927147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c807c-b1cd-47e8-a1a4-424f751179f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:37.927581 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c807c-b1cd-47e8-a1a4-424f751179f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:37.928370 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c807c-b1cd-47e8-a1a4-424f751179f1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:37.928687 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c807c-b1cd-47e8-a1a4-424f751179f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:37.930082 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c807c-b1cd-47e8-a1a4-424f751179f1 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:37.940462 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 47/87] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 3 04:53:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:37.941923 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c807c-b1cd-47e8-a1a4-424f751179f1 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:37.981714 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:37.996988 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:38.041671 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 41/88] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:53:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:53:38.042688 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad033dd8-c0e9-4bd7-b73c-70e1c589ba76 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:38.069626 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c807c-b1cd-47e8-a1a4-424f751179f1 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:38.109201 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3ef16-38c3-44ed-b0df-693b9e97111f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:38.109901 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3ef16-38c3-44ed-b0df-693b9e97111f None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:38.110194 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3ef16-38c3-44ed-b0df-693b9e97111f None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:38.110484 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3ef16-38c3-44ed-b0df-693b9e97111f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:38.111298 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3ef16-38c3-44ed-b0df-693b9e97111f None admin] RBAC: Adding query filter params (role.id=b0e0084bb31348a582616b1421f3b10c, scope.project.id=1952481417bf4a2cad0b08d5a7676b61, user.id=b2cf0210955141068b832fc376bdc5ab) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:38.111659 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3ef16-38c3-44ed-b0df-693b9e97111f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:38.113172 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3ef16-38c3-44ed-b0df-693b9e97111f None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:38.120221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 48/89] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:53:37 2022] GET /identity/v3/role_assignments?user.id=b2cf0210955141068b832fc376bdc5ab&scope.project.id=1952481417bf4a2cad0b08d5a7676b61&role.id=b0e0084bb31348a582616b1421f3b10c => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:38.121486 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3ef16-38c3-44ed-b0df-693b9e97111f None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:38.176907 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad033dd8-c0e9-4bd7-b73c-70e1c589ba76 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:38.238400 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78c3e2-4b96-4741-922a-fe9f6ce5b663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:38.239094 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78c3e2-4b96-4741-922a-fe9f6ce5b663 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:38.239383 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78c3e2-4b96-4741-922a-fe9f6ce5b663 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:38.239668 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78c3e2-4b96-4741-922a-fe9f6ce5b663 None admin] PATH_INFO: `/v3/projects/1952481417bf4a2cad0b08d5a7676b61` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:38.240947 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af78c3e2-4b96-4741-922a-fe9f6ce5b663 None admin] RBAC: Authorizing `identity:get_project(project_id=1952481417bf4a2cad0b08d5a7676b61)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:38.242652 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af78c3e2-4b96-4741-922a-fe9f6ce5b663 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:38.245306 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 42/90] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:53:38 2022] GET /identity/v3/projects/1952481417bf4a2cad0b08d5a7676b61 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:38.249674 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78c3e2-4b96-4741-922a-fe9f6ce5b663 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:38.311071 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:38.334268 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:38.342437 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3ef16-38c3-44ed-b0df-693b9e97111f None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:38.410425 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079c993-5b5d-47d5-8edc-63995f4af0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:38.411773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079c993-5b5d-47d5-8edc-63995f4af0dc None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:38.411773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079c993-5b5d-47d5-8edc-63995f4af0dc None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:38.411773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079c993-5b5d-47d5-8edc-63995f4af0dc None admin] PATH_INFO: `/v3/users/b2cf0210955141068b832fc376bdc5ab` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:38.412760 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5079c993-5b5d-47d5-8edc-63995f4af0dc None admin] RBAC: Authorizing `identity:get_user(user_id=b2cf0210955141068b832fc376bdc5ab)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:38.414207 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5079c993-5b5d-47d5-8edc-63995f4af0dc None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:38.416325 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 49/91] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:53:38 2022] GET /identity/v3/users/b2cf0210955141068b832fc376bdc5ab => generated 275 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:38.417334 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079c993-5b5d-47d5-8edc-63995f4af0dc None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:38.422886 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78c3e2-4b96-4741-922a-fe9f6ce5b663 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:38.476399 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82da1df-ea6f-4328-b537-905b1cbe0450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:38.477042 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82da1df-ea6f-4328-b537-905b1cbe0450 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:38.477329 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82da1df-ea6f-4328-b537-905b1cbe0450 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:38.477608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82da1df-ea6f-4328-b537-905b1cbe0450 None admin] PATH_INFO: `/v3/projects/1952481417bf4a2cad0b08d5a7676b61` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:38.478972 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82da1df-ea6f-4328-b537-905b1cbe0450 None admin] RBAC: Authorizing `identity:get_project(project_id=1952481417bf4a2cad0b08d5a7676b61)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:38.480428 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82da1df-ea6f-4328-b537-905b1cbe0450 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:38.482205 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 43/92] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:53:38 2022] GET /identity/v3/projects/1952481417bf4a2cad0b08d5a7676b61 => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:38.483100 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82da1df-ea6f-4328-b537-905b1cbe0450 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:38.614414 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82da1df-ea6f-4328-b537-905b1cbe0450 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:38.635192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:38.643799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079c993-5b5d-47d5-8edc-63995f4af0dc None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:38.669405 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1297c4-2246-43b0-9dea-70700a992853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:38.670290 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1297c4-2246-43b0-9dea-70700a992853 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:38.670646 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1297c4-2246-43b0-9dea-70700a992853 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:38.670949 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1297c4-2246-43b0-9dea-70700a992853 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:38.710735 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc450600-3d5c-44e1-9834-8e4c22e8a627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:38.711432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc450600-3d5c-44e1-9834-8e4c22e8a627 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:38.711887 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc450600-3d5c-44e1-9834-8e4c22e8a627 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:38.712187 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc450600-3d5c-44e1-9834-8e4c22e8a627 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:38.713610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc450600-3d5c-44e1-9834-8e4c22e8a627 None admin] RBAC: Adding query filter params (role.id=561445e71aff40a0af7b01c539d3cfd9, scope.project.id=1952481417bf4a2cad0b08d5a7676b61, user.id=b2cf0210955141068b832fc376bdc5ab) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:38.713931 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc450600-3d5c-44e1-9834-8e4c22e8a627 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:38.715610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc450600-3d5c-44e1-9834-8e4c22e8a627 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:38.727150 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 50/93] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:53:38 2022] GET /identity/v3/role_assignments?user.id=b2cf0210955141068b832fc376bdc5ab&scope.project.id=1952481417bf4a2cad0b08d5a7676b61&role.id=561445e71aff40a0af7b01c539d3cfd9 => generated 259 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:38.735816 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc450600-3d5c-44e1-9834-8e4c22e8a627 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:38.794267 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1297c4-2246-43b0-9dea-70700a992853 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:38.795872 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1297c4-2246-43b0-9dea-70700a992853 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:38.840971 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:39.007162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc450600-3d5c-44e1-9834-8e4c22e8a627 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:39.035194 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 44/94] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:53:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:53:39.042877 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1297c4-2246-43b0-9dea-70700a992853 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:39.054213 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:39.061680 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19d6c7-edb2-4ad4-9bad-8e367d7df8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:39.062325 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19d6c7-edb2-4ad4-9bad-8e367d7df8d7 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:39.062615 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19d6c7-edb2-4ad4-9bad-8e367d7df8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:39.063102 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19d6c7-edb2-4ad4-9bad-8e367d7df8d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:39.064285 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19d6c7-edb2-4ad4-9bad-8e367d7df8d7 None admin] RBAC: Adding query filter params (role.id=e39d4ecea4c64d5794e1079c9be3ad36, scope.project.id=1952481417bf4a2cad0b08d5a7676b61, user.id=d53b9368b2734205a2bdcd0bc2796161) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:39.076298 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19d6c7-edb2-4ad4-9bad-8e367d7df8d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:39.076298 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19d6c7-edb2-4ad4-9bad-8e367d7df8d7 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:39.076298 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 51/95] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:53:38 2022] GET /identity/v3/role_assignments?user.id=d53b9368b2734205a2bdcd0bc2796161&scope.project.id=1952481417bf4a2cad0b08d5a7676b61&role.id=e39d4ecea4c64d5794e1079c9be3ad36 => generated 259 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:39.218057 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1297c4-2246-43b0-9dea-70700a992853 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:39.268787 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217a311-7f3b-485f-bdc8-930100039269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:39.269369 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217a311-7f3b-485f-bdc8-930100039269 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:39.269657 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217a311-7f3b-485f-bdc8-930100039269 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:39.269938 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217a311-7f3b-485f-bdc8-930100039269 None admin] PATH_INFO: `/v3/projects/4b2abb3ea36a4542819443c464ad5a0d` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:39.271146 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217a311-7f3b-485f-bdc8-930100039269 None admin] RBAC: Authorizing `identity:get_project(project_id=4b2abb3ea36a4542819443c464ad5a0d)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:39.272636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217a311-7f3b-485f-bdc8-930100039269 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:39.274573 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 45/96] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:53:39 2022] GET /identity/v3/projects/4b2abb3ea36a4542819443c464ad5a0d => generated 312 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:39.280654 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19d6c7-edb2-4ad4-9bad-8e367d7df8d7 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:39.384591 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:39.391356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19d6c7-edb2-4ad4-9bad-8e367d7df8d7 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:39.426824 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c7d54-402c-440f-b6aa-b8f3b3109b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:39.427474 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c7d54-402c-440f-b6aa-b8f3b3109b88 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:39.427759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c7d54-402c-440f-b6aa-b8f3b3109b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:39.428043 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c7d54-402c-440f-b6aa-b8f3b3109b88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:39.428836 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c7d54-402c-440f-b6aa-b8f3b3109b88 None admin] RBAC: Adding query filter params (role.id=b0e0084bb31348a582616b1421f3b10c, scope.project.id=4b2abb3ea36a4542819443c464ad5a0d, user.id=b2cf0210955141068b832fc376bdc5ab) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:39.429149 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c7d54-402c-440f-b6aa-b8f3b3109b88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:39.430612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c7d54-402c-440f-b6aa-b8f3b3109b88 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:39.436572 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 52/97] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:53:39 2022] GET /identity/v3/role_assignments?user.id=b2cf0210955141068b832fc376bdc5ab&scope.project.id=4b2abb3ea36a4542819443c464ad5a0d&role.id=b0e0084bb31348a582616b1421f3b10c => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:40.518379 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a41788-f8da-4616-8d56-8ad164b204d1 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:40.519442 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a41788-f8da-4616-8d56-8ad164b204d1 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:40.519934 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a41788-f8da-4616-8d56-8ad164b204d1 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:40.521512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 46/98] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:40.527109 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1eafc4-4003-4310-9b61-6e291d77f867 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:40.527939 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1eafc4-4003-4310-9b61-6e291d77f867 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:40.528440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1eafc4-4003-4310-9b61-6e291d77f867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:40.548197 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-6a1eafc4-4003-4310-9b61-6e291d77f867 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:40.549762 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-6a1eafc4-4003-4310-9b61-6e291d77f867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:40.645637 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 53/99] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:53:40.676574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a41788-f8da-4616-8d56-8ad164b204d1 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:40.772043 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:40.786290 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2c33c-3bee-4ef2-8b75-3fb495b5c039 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:40.792464 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2c33c-3bee-4ef2-8b75-3fb495b5c039 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:40.792464 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2c33c-3bee-4ef2-8b75-3fb495b5c039 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:40.792464 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 54/100] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:40.794088 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccb280-05e2-4942-a92c-ee6717d4cf18 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:40.794599 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccb280-05e2-4942-a92c-ee6717d4cf18 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:40.795244 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccb280-05e2-4942-a92c-ee6717d4cf18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:40.812531 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a41788-f8da-4616-8d56-8ad164b204d1 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:40.824634 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-a0ccb280-05e2-4942-a92c-ee6717d4cf18 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:40.826151 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-a0ccb280-05e2-4942-a92c-ee6717d4cf18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:40.874638 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed360c99-6694-43cf-a184-5d54af327504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:40.875584 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed360c99-6694-43cf-a184-5d54af327504 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:40.876086 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed360c99-6694-43cf-a184-5d54af327504 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:40.876621 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed360c99-6694-43cf-a184-5d54af327504 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:40.888844 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed360c99-6694-43cf-a184-5d54af327504 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:53:40.895839 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-ed360c99-6694-43cf-a184-5d54af327504 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 04:53:40.895839 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:53:40.895839 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:53:40.895839 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:53:40.895839 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:53:40.895839 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:53:40.895839 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:53:40.895839 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:53:40.895839 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:53:40.895839 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:53:40.895839 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:53:40.895839 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:53:40.895839 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:53:40.895839 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:53:40.895839 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:53:40.895839 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:53:40.896710 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:53:40.896710 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:53:40.896710 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:53:40.896710 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:53:40.896710 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:53:40.896710 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:53:40.896710 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:53:40.896710 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:53:40.896710 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:53:40.896710 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:53:40.896710 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:53:40.896710 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:53:40.896710 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:53:40.896710 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:53:40.896710 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:53:40.896710 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:53:40.897471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:53:40.897471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:53:40.897471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:53:40.897471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:53:40.897471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:53:40.897471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:53:40.897471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:53:40.897471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:53:40.897471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:53:40.897471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:53:40.897471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 04:53:40.897471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:53:40.900239 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 47/101] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:53:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:53:40.906567 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed360c99-6694-43cf-a184-5d54af327504 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:40.964021 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 55/102] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:53:41.024145 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ccb280-05e2-4942-a92c-ee6717d4cf18 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:41.054586 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:41.073807 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed360c99-6694-43cf-a184-5d54af327504 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:41.107209 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:41.119233 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b251e7f-70b4-444b-8234-febf3fa3e9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:41.120577 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b251e7f-70b4-444b-8234-febf3fa3e9c7 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:41.121061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b251e7f-70b4-444b-8234-febf3fa3e9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:41.121501 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b251e7f-70b4-444b-8234-febf3fa3e9c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:41.122655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b251e7f-70b4-444b-8234-febf3fa3e9c7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:41.124857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b251e7f-70b4-444b-8234-febf3fa3e9c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:41.129546 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b251e7f-70b4-444b-8234-febf3fa3e9c7 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:41.144798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 48/103] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 3 04:53:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:53:41.146328 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfc270-a636-4370-9d78-320035435b1f None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:41.152200 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfc270-a636-4370-9d78-320035435b1f None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:41.152496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfc270-a636-4370-9d78-320035435b1f None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:41.153843 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 49/104] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:41.154650 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfc270-a636-4370-9d78-320035435b1f None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:41.206323 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ccb280-05e2-4942-a92c-ee6717d4cf18 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:41.261388 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9bd1a-2cff-4e6f-a242-30d667b69559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:41.263595 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9bd1a-2cff-4e6f-a242-30d667b69559 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:41.265636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9bd1a-2cff-4e6f-a242-30d667b69559 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:41.266124 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9bd1a-2cff-4e6f-a242-30d667b69559 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:41.306918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac9bd1a-2cff-4e6f-a242-30d667b69559 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:53:41.306918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-1ac9bd1a-2cff-4e6f-a242-30d667b69559 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:53:41.306918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:53:41.306918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:53:41.306918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:53:41.306918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:53:41.306918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:53:41.306918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:53:41.306918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:53:41.306918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:53:41.306918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:53:41.306918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:53:41.306918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:53:41.306918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:53:41.307773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:53:41.307773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:53:41.307773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:53:41.307773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:53:41.307773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:53:41.307773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:53:41.307773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:53:41.307773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:53:41.307773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:53:41.307773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:53:41.307773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:53:41.307773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:53:41.307773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:53:41.307773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:53:41.307773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:53:41.307773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:53:41.307773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:53:41.308610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:53:41.308610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:53:41.308610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:53:41.308610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:53:41.308610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:53:41.308610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:53:41.308610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:53:41.308610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:53:41.308610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:53:41.308610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:53:41.308610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:53:41.308610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:53:41.308610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:53:41.308610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:53:41.308610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 56/105] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:53:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:53:41.309463 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba471e7b-e46c-4e27-897d-9ca9a29dd2bf None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:41.309463 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba471e7b-e46c-4e27-897d-9ca9a29dd2bf None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:41.309463 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba471e7b-e46c-4e27-897d-9ca9a29dd2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:41.337617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfc270-a636-4370-9d78-320035435b1f None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:41.351980 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-ba471e7b-e46c-4e27-897d-9ca9a29dd2bf None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:41.353194 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-ba471e7b-e46c-4e27-897d-9ca9a29dd2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:41.415919 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d94b1-942f-4e1f-8708-0be0f33ef980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:41.417109 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d94b1-942f-4e1f-8708-0be0f33ef980 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:41.417639 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d94b1-942f-4e1f-8708-0be0f33ef980 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:41.418143 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d94b1-942f-4e1f-8708-0be0f33ef980 None admin] PATH_INFO: `/v3/users/4037c80bf46a404da45ebfd0596d5a12` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:41.437131 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 57/106] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:53:41.437880 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba471e7b-e46c-4e27-897d-9ca9a29dd2bf None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:41.447550 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d94b1-942f-4e1f-8708-0be0f33ef980 None admin] RBAC: Authorizing `identity:get_user(user_id=4037c80bf46a404da45ebfd0596d5a12)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:41.449296 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d94b1-942f-4e1f-8708-0be0f33ef980 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:41.449554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:41.452035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 50/107] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:53:41 2022] GET /identity/v3/users/4037c80bf46a404da45ebfd0596d5a12 => generated 289 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:41.457689 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d94b1-942f-4e1f-8708-0be0f33ef980 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:41.545517 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba471e7b-e46c-4e27-897d-9ca9a29dd2bf None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:41.563594 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:41.582874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd22e1-745c-499b-80ed-8f52cff1f30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:41.583562 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd22e1-745c-499b-80ed-8f52cff1f30b None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:41.584051 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd22e1-745c-499b-80ed-8f52cff1f30b None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:41.584504 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd22e1-745c-499b-80ed-8f52cff1f30b None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:41.585407 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fd22e1-745c-499b-80ed-8f52cff1f30b None admin] RBAC: Adding query filter params (name=member) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:41.585860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fd22e1-745c-499b-80ed-8f52cff1f30b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:41.587454 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fd22e1-745c-499b-80ed-8f52cff1f30b None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:41.597264 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 58/108] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 3 04:53:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:41.598335 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd22e1-745c-499b-80ed-8f52cff1f30b None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:41.679925 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:41.712568 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd22e1-745c-499b-80ed-8f52cff1f30b None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:41.726596 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d94b1-942f-4e1f-8708-0be0f33ef980 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:41.762449 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c517451-f27e-4cb0-ac2d-7596ccdaa905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:41.764135 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c517451-f27e-4cb0-ac2d-7596ccdaa905 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:41.764730 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c517451-f27e-4cb0-ac2d-7596ccdaa905 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:41.766233 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c517451-f27e-4cb0-ac2d-7596ccdaa905 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:41.767833 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c517451-f27e-4cb0-ac2d-7596ccdaa905 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:41.773196 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c517451-f27e-4cb0-ac2d-7596ccdaa905 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:41.776948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 59/109] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:53:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:41.777950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c517451-f27e-4cb0-ac2d-7596ccdaa905 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:41.808748 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b35d1-a3d9-483f-b7ed-6d80b90a62a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:41.809693 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b35d1-a3d9-483f-b7ed-6d80b90a62a6 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:41.810128 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b35d1-a3d9-483f-b7ed-6d80b90a62a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:41.810551 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b35d1-a3d9-483f-b7ed-6d80b90a62a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:41.935141 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b35d1-a3d9-483f-b7ed-6d80b90a62a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:41.936539 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b35d1-a3d9-483f-b7ed-6d80b90a62a6 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:41.939442 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c517451-f27e-4cb0-ac2d-7596ccdaa905 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:41.976988 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5660e673-a3c5-4091-b98f-00fc3154b4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:41.977639 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660e673-a3c5-4091-b98f-00fc3154b4ab None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:41.977929 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660e673-a3c5-4091-b98f-00fc3154b4ab None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:41.978216 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660e673-a3c5-4091-b98f-00fc3154b4ab None admin] PATH_INFO: `/v3/users/b2cf0210955141068b832fc376bdc5ab` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:41.980155 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5660e673-a3c5-4091-b98f-00fc3154b4ab None admin] RBAC: Authorizing `identity:get_user(user_id=b2cf0210955141068b832fc376bdc5ab)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:41.981639 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5660e673-a3c5-4091-b98f-00fc3154b4ab None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:41.983487 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 60/110] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:53:41 2022] GET /identity/v3/users/b2cf0210955141068b832fc376bdc5ab => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:41.984080 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5660e673-a3c5-4091-b98f-00fc3154b4ab None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:41.993707 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:41.995352 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:42.042952 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 51/111] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:53:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:53:42.044082 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530f12a-423b-4cda-bdaf-7ccd884b8109 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:42.044383 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530f12a-423b-4cda-bdaf-7ccd884b8109 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:42.044717 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530f12a-423b-4cda-bdaf-7ccd884b8109 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:42.046118 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 52/112] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:42.051943 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f5384-0fca-46c1-ae85-747832a3d662 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:42.052241 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f5384-0fca-46c1-ae85-747832a3d662 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:42.052532 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f5384-0fca-46c1-ae85-747832a3d662 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:42.054894 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 53/113] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:42.054894 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f5384-0fca-46c1-ae85-747832a3d662 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:42.097914 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5660e673-a3c5-4091-b98f-00fc3154b4ab None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:42.137612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1942fc6-116b-4435-a769-04cb1f5a46cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:42.138680 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1942fc6-116b-4435-a769-04cb1f5a46cf None admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:42.139206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1942fc6-116b-4435-a769-04cb1f5a46cf None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:42.139679 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1942fc6-116b-4435-a769-04cb1f5a46cf None admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:42.140722 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1942fc6-116b-4435-a769-04cb1f5a46cf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:42.142532 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1942fc6-116b-4435-a769-04cb1f5a46cf None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:42.162669 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f5384-0fca-46c1-ae85-747832a3d662 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:42.198297 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad4284-8d0b-46f5-a468-39aa8f043535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:42.198905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad4284-8d0b-46f5-a468-39aa8f043535 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:42.199193 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad4284-8d0b-46f5-a468-39aa8f043535 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:42.199473 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad4284-8d0b-46f5-a468-39aa8f043535 None admin] PATH_INFO: `/v3/projects/1952481417bf4a2cad0b08d5a7676b61` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:42.200646 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad4284-8d0b-46f5-a468-39aa8f043535 None admin] RBAC: Authorizing `identity:get_project(project_id=1952481417bf4a2cad0b08d5a7676b61)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:42.202113 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad4284-8d0b-46f5-a468-39aa8f043535 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:42.203812 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 54/114] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:53:42 2022] GET /identity/v3/projects/1952481417bf4a2cad0b08d5a7676b61 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:42.204613 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad4284-8d0b-46f5-a468-39aa8f043535 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:42.224464 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 61/115] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Aug 3 04:53:41 2022] POST /identity/v3/projects => generated 302 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:53:42.225718 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f805a4-389d-492a-a32f-7908a91b4a57 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:42.226015 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f805a4-389d-492a-a32f-7908a91b4a57 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:42.226304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f805a4-389d-492a-a32f-7908a91b4a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:42.229562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:42.245737 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-b5f805a4-389d-492a-a32f-7908a91b4a57 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:42.247054 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-b5f805a4-389d-492a-a32f-7908a91b4a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:42.308299 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:42.323339 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 62/116] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:53:42.324812 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d382141-3dec-4aef-93c5-e8ecfc61842b None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:42.325111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d382141-3dec-4aef-93c5-e8ecfc61842b None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:42.325398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d382141-3dec-4aef-93c5-e8ecfc61842b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:42.342870 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad4284-8d0b-46f5-a468-39aa8f043535 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:42.357775 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-9d382141-3dec-4aef-93c5-e8ecfc61842b None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:42.359076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-9d382141-3dec-4aef-93c5-e8ecfc61842b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:42.384637 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e30d1-4d92-4e28-9e7a-061a28a93b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:42.385681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e30d1-4d92-4e28-9e7a-061a28a93b51 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:42.386146 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e30d1-4d92-4e28-9e7a-061a28a93b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:42.386433 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e30d1-4d92-4e28-9e7a-061a28a93b51 None admin] PATH_INFO: `/v3/projects/1952481417bf4a2cad0b08d5a7676b61` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:42.387650 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e30d1-4d92-4e28-9e7a-061a28a93b51 None admin] RBAC: Authorizing `identity:get_project(project_id=1952481417bf4a2cad0b08d5a7676b61)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:42.389115 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e30d1-4d92-4e28-9e7a-061a28a93b51 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:42.390794 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 55/117] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:53:42 2022] GET /identity/v3/projects/1952481417bf4a2cad0b08d5a7676b61 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:42.391572 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e30d1-4d92-4e28-9e7a-061a28a93b51 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:42.444455 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 63/118] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:53:42.444455 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1b815-d324-42b5-b757-8a82344af47b None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:42.444455 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1b815-d324-42b5-b757-8a82344af47b None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:42.444455 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1b815-d324-42b5-b757-8a82344af47b None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:42.444455 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 64/119] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:42.445044 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1b815-d324-42b5-b757-8a82344af47b None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:42.507156 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:42.520145 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e30d1-4d92-4e28-9e7a-061a28a93b51 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:42.558141 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a440508-fe78-426c-9986-de72cf8ce201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:42.558830 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a440508-fe78-426c-9986-de72cf8ce201 None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:42.559139 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a440508-fe78-426c-9986-de72cf8ce201 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:42.559419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a440508-fe78-426c-9986-de72cf8ce201 None admin] PATH_INFO: `/v3/projects/1952481417bf4a2cad0b08d5a7676b61/users/b2cf0210955141068b832fc376bdc5ab/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:42.561404 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a440508-fe78-426c-9986-de72cf8ce201 None admin] RBAC: Authorizing `identity:create_grant(project_id=1952481417bf4a2cad0b08d5a7676b61, user_id=b2cf0210955141068b832fc376bdc5ab, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:42.562967 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a440508-fe78-426c-9986-de72cf8ce201 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:42.568986 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1b815-d324-42b5-b757-8a82344af47b None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:42.589719 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:42.607598 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60881d78-b2d2-4e34-993a-3e8be1ab0635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:42.608255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60881d78-b2d2-4e34-993a-3e8be1ab0635 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:42.608542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60881d78-b2d2-4e34-993a-3e8be1ab0635 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:42.608826 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60881d78-b2d2-4e34-993a-3e8be1ab0635 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:42.611863 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 56/120] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 3 04:53:42 2022] PUT /identity/v3/projects/1952481417bf4a2cad0b08d5a7676b61/users/b2cf0210955141068b832fc376bdc5ab/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 04:53:42.612652 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a440508-fe78-426c-9986-de72cf8ce201 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:42.634239 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60881d78-b2d2-4e34-993a-3e8be1ab0635 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:53:42.639993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-60881d78-b2d2-4e34-993a-3e8be1ab0635 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 04:53:42.639993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:53:42.639993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:53:42.639993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:53:42.639993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:53:42.639993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:53:42.639993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:53:42.639993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:53:42.639993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:53:42.639993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:53:42.639993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:53:42.639993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:53:42.639993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:53:42.639993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:53:42.639993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:53:42.639993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:53:42.640950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:53:42.640950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:53:42.640950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:53:42.640950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:53:42.640950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:53:42.640950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:53:42.640950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:53:42.640950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:53:42.640950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:53:42.640950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:53:42.640950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:53:42.640950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:53:42.640950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:53:42.640950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:53:42.640950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:53:42.640950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:53:42.641756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:53:42.641756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:53:42.641756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:53:42.641756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:53:42.641756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:53:42.641756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:53:42.641756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:53:42.641756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:53:42.641756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:53:42.641756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:53:42.641756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 04:53:42.641756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:53:42.641756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 65/121] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 3 04:53:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:53:42.643599 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378471f5-2629-4838-88c2-bb7e048f41c9 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:42.644455 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378471f5-2629-4838-88c2-bb7e048f41c9 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:42.644748 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378471f5-2629-4838-88c2-bb7e048f41c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:42.668401 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-378471f5-2629-4838-88c2-bb7e048f41c9 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:42.670430 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-378471f5-2629-4838-88c2-bb7e048f41c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:42.763759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 66/122] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:53:42.763759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378471f5-2629-4838-88c2-bb7e048f41c9 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:42.768343 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a440508-fe78-426c-9986-de72cf8ce201 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:42.826900 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cace460-6e4b-4f79-9d6e-8cf1236d81f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:42.832419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cace460-6e4b-4f79-9d6e-8cf1236d81f3 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:42.832419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cace460-6e4b-4f79-9d6e-8cf1236d81f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:42.833492 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cace460-6e4b-4f79-9d6e-8cf1236d81f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:42.834068 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cace460-6e4b-4f79-9d6e-8cf1236d81f3 None admin] RBAC: Adding query filter params (role.id=d7a7654a12dd4bd38542203bde537922, scope.project.id=1952481417bf4a2cad0b08d5a7676b61, user.id=4037c80bf46a404da45ebfd0596d5a12) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:42.834395 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cace460-6e4b-4f79-9d6e-8cf1236d81f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:42.836913 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cace460-6e4b-4f79-9d6e-8cf1236d81f3 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:42.845755 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 57/123] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:53:42 2022] GET /identity/v3/role_assignments?user.id=4037c80bf46a404da45ebfd0596d5a12&scope.project.id=1952481417bf4a2cad0b08d5a7676b61&role.id=d7a7654a12dd4bd38542203bde537922 => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:42.846485 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cace460-6e4b-4f79-9d6e-8cf1236d81f3 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:42.866495 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:42.890582 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378471f5-2629-4838-88c2-bb7e048f41c9 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:42.903038 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:42.950871 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45061418-19ba-4439-b318-4e9965e1a590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:42.950871 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45061418-19ba-4439-b318-4e9965e1a590 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:42.950871 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45061418-19ba-4439-b318-4e9965e1a590 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:42.950871 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45061418-19ba-4439-b318-4e9965e1a590 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:42.974750 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45061418-19ba-4439-b318-4e9965e1a590 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:53:42.985828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-45061418-19ba-4439-b318-4e9965e1a590 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:53:42.985828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:53:42.985828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:53:42.985828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:53:42.985828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:53:42.985828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:53:42.985828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:53:42.985828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:53:42.985828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:53:42.985828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:53:42.985828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:53:42.985828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:53:42.985828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:53:42.985828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:53:42.985828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:53:42.985828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:53:42.986792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:53:42.986792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:53:42.986792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:53:42.986792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:53:42.986792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:53:42.986792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:53:42.986792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:53:42.986792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:53:42.986792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:53:42.986792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:53:42.986792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:53:42.986792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:53:42.986792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:53:42.986792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:53:42.986792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:53:42.986792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:53:42.988173 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:53:42.988173 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:53:42.988173 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:53:42.988173 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:53:42.988173 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:53:42.988173 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:53:42.988173 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:53:42.988173 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:53:42.988173 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:53:42.988173 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:53:42.988173 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:53:42.988173 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:53:42.992286 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 67/124] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:53:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:53:42.993328 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45061418-19ba-4439-b318-4e9965e1a590 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:43.008340 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cace460-6e4b-4f79-9d6e-8cf1236d81f3 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:43.077867 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e83bd-cc8a-47de-a7ae-a4275e703825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:43.078424 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e83bd-cc8a-47de-a7ae-a4275e703825 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:43.078714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e83bd-cc8a-47de-a7ae-a4275e703825 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:43.079038 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e83bd-cc8a-47de-a7ae-a4275e703825 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:43.079808 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e83bd-cc8a-47de-a7ae-a4275e703825 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:43.080089 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e83bd-cc8a-47de-a7ae-a4275e703825 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:43.081555 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e83bd-cc8a-47de-a7ae-a4275e703825 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:43.100076 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 58/125] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:53:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:43.100953 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e83bd-cc8a-47de-a7ae-a4275e703825 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:43.188157 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45061418-19ba-4439-b318-4e9965e1a590 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:43.236224 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:43.251750 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b0860-9f2f-4c5f-bbb9-93c09a9ec6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:43.254535 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b0860-9f2f-4c5f-bbb9-93c09a9ec6d8 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:43.254535 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b0860-9f2f-4c5f-bbb9-93c09a9ec6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:43.254535 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b0860-9f2f-4c5f-bbb9-93c09a9ec6d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:43.285327 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e83bd-cc8a-47de-a7ae-a4275e703825 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:43.286169 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481b0860-9f2f-4c5f-bbb9-93c09a9ec6d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:53:43.292147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-481b0860-9f2f-4c5f-bbb9-93c09a9ec6d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:53:43.292147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:53:43.292147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:53:43.292147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:53:43.292147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:53:43.292147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:53:43.292147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:53:43.292147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:53:43.292147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:53:43.292147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:53:43.292147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:53:43.292147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:53:43.292147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:53:43.292147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:53:43.292147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:53:43.292147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:53:43.292985 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:53:43.292985 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:53:43.292985 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:53:43.292985 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:53:43.292985 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:53:43.292985 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:53:43.292985 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:53:43.292985 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:53:43.292985 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:53:43.292985 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:53:43.292985 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:53:43.292985 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:53:43.292985 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:53:43.292985 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:53:43.292985 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:53:43.292985 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:53:43.293772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:53:43.293772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:53:43.293772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:53:43.293772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:53:43.293772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:53:43.293772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:53:43.293772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:53:43.293772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:53:43.293772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:53:43.293772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:53:43.293772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:53:43.293772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:53:43.302985 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 68/126] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:53:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:53:43.304116 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b0860-9f2f-4c5f-bbb9-93c09a9ec6d8 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:43.329172 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51918a-148b-4766-b692-01b707803d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:43.330628 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51918a-148b-4766-b692-01b707803d64 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:43.330941 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51918a-148b-4766-b692-01b707803d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:43.331257 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51918a-148b-4766-b692-01b707803d64 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:43.334424 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51918a-148b-4766-b692-01b707803d64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:43.336106 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51918a-148b-4766-b692-01b707803d64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:43.337618 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51918a-148b-4766-b692-01b707803d64 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:43.355778 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 59/127] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:53:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:43.356633 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51918a-148b-4766-b692-01b707803d64 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:43.360797 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:43.481850 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51918a-148b-4766-b692-01b707803d64 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:43.486304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b0860-9f2f-4c5f-bbb9-93c09a9ec6d8 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:43.521747 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db98466a-b172-4e78-9b4c-45f0eca650d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:43.522498 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98466a-b172-4e78-9b4c-45f0eca650d4 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:43.522790 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98466a-b172-4e78-9b4c-45f0eca650d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:43.523101 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98466a-b172-4e78-9b4c-45f0eca650d4 None admin] PATH_INFO: `/v3/users/b2cf0210955141068b832fc376bdc5ab` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:43.524296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db98466a-b172-4e78-9b4c-45f0eca650d4 None admin] RBAC: Authorizing `identity:get_user(user_id=b2cf0210955141068b832fc376bdc5ab)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:43.525264 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d60aa-2aac-4be4-844c-2bd57c513fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:43.525786 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db98466a-b172-4e78-9b4c-45f0eca650d4 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:43.525889 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d60aa-2aac-4be4-844c-2bd57c513fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:43.526169 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d60aa-2aac-4be4-844c-2bd57c513fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:43.526719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d60aa-2aac-4be4-844c-2bd57c513fb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:43.527495 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d60aa-2aac-4be4-844c-2bd57c513fb9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:43.527756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 69/128] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:53:43 2022] GET /identity/v3/users/b2cf0210955141068b832fc376bdc5ab => generated 275 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:43.528131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d60aa-2aac-4be4-844c-2bd57c513fb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:43.528666 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db98466a-b172-4e78-9b4c-45f0eca650d4 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:43.529603 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d60aa-2aac-4be4-844c-2bd57c513fb9 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:43.544008 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 60/129] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 3 04:53:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:43.545034 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d60aa-2aac-4be4-844c-2bd57c513fb9 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:43.559247 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:43.649870 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:43.657708 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db98466a-b172-4e78-9b4c-45f0eca650d4 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:43.699548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338b0d2-a902-4383-8d62-bf885ebd7a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:43.700208 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338b0d2-a902-4383-8d62-bf885ebd7a1e None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:43.700497 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338b0d2-a902-4383-8d62-bf885ebd7a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:43.700777 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338b0d2-a902-4383-8d62-bf885ebd7a1e None admin] PATH_INFO: `/v3/users/d53b9368b2734205a2bdcd0bc2796161` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:43.703705 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338b0d2-a902-4383-8d62-bf885ebd7a1e None admin] RBAC: Authorizing `identity:get_user(user_id=d53b9368b2734205a2bdcd0bc2796161)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:43.705503 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338b0d2-a902-4383-8d62-bf885ebd7a1e None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:43.708416 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 70/130] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:53:43 2022] GET /identity/v3/users/d53b9368b2734205a2bdcd0bc2796161 => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:43.709618 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338b0d2-a902-4383-8d62-bf885ebd7a1e None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:43.729697 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d60aa-2aac-4be4-844c-2bd57c513fb9 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:43.770604 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e96be-59e2-42cf-bd47-8555c77120e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:43.771448 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e96be-59e2-42cf-bd47-8555c77120e6 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:43.771740 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e96be-59e2-42cf-bd47-8555c77120e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:43.772028 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e96be-59e2-42cf-bd47-8555c77120e6 None admin] PATH_INFO: `/v3/projects/1952481417bf4a2cad0b08d5a7676b61` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:43.778865 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7e96be-59e2-42cf-bd47-8555c77120e6 None admin] RBAC: Authorizing `identity:get_project(project_id=1952481417bf4a2cad0b08d5a7676b61)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:43.778865 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7e96be-59e2-42cf-bd47-8555c77120e6 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:43.778865 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 61/131] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:53:43 2022] GET /identity/v3/projects/1952481417bf4a2cad0b08d5a7676b61 => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:43.778865 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e96be-59e2-42cf-bd47-8555c77120e6 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:43.869315 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:43.885429 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338b0d2-a902-4383-8d62-bf885ebd7a1e None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:43.916411 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e96be-59e2-42cf-bd47-8555c77120e6 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:43.924631 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb04a6-866b-4e3d-af75-e67166eb3b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:43.926295 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb04a6-866b-4e3d-af75-e67166eb3b0a None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:43.926805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb04a6-866b-4e3d-af75-e67166eb3b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:43.927629 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb04a6-866b-4e3d-af75-e67166eb3b0a None admin] PATH_INFO: `/v3/users/b2cf0210955141068b832fc376bdc5ab` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:43.929286 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb04a6-866b-4e3d-af75-e67166eb3b0a None admin] RBAC: Authorizing `identity:get_user(user_id=b2cf0210955141068b832fc376bdc5ab)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:43.931269 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb04a6-866b-4e3d-af75-e67166eb3b0a None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:43.933399 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 71/132] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:53:43 2022] GET /identity/v3/users/b2cf0210955141068b832fc376bdc5ab => generated 275 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:43.934356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb04a6-866b-4e3d-af75-e67166eb3b0a None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:43.973159 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199309f0-9894-44ef-b3e8-1f1db412e940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:43.973850 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199309f0-9894-44ef-b3e8-1f1db412e940 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:43.974142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199309f0-9894-44ef-b3e8-1f1db412e940 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:43.974431 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199309f0-9894-44ef-b3e8-1f1db412e940 None admin] PATH_INFO: `/v3/projects/1952481417bf4a2cad0b08d5a7676b61` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:43.978589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199309f0-9894-44ef-b3e8-1f1db412e940 None admin] RBAC: Authorizing `identity:get_project(project_id=1952481417bf4a2cad0b08d5a7676b61)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:43.980159 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199309f0-9894-44ef-b3e8-1f1db412e940 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:43.985949 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 62/133] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:53:43 2022] GET /identity/v3/projects/1952481417bf4a2cad0b08d5a7676b61 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:43.986814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199309f0-9894-44ef-b3e8-1f1db412e940 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:43.998832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:44.078261 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb04a6-866b-4e3d-af75-e67166eb3b0a None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:44.118685 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed51cd6-66b4-4fba-b301-7862648365bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:44.119585 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed51cd6-66b4-4fba-b301-7862648365bc None admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:44.120338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed51cd6-66b4-4fba-b301-7862648365bc None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:44.120633 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed51cd6-66b4-4fba-b301-7862648365bc None admin] PATH_INFO: `/v3/projects/1952481417bf4a2cad0b08d5a7676b61/users/b2cf0210955141068b832fc376bdc5ab/roles/561445e71aff40a0af7b01c539d3cfd9` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:44.125952 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed51cd6-66b4-4fba-b301-7862648365bc None admin] RBAC: Authorizing `identity:create_grant(project_id=1952481417bf4a2cad0b08d5a7676b61, user_id=b2cf0210955141068b832fc376bdc5ab, role_id=561445e71aff40a0af7b01c539d3cfd9)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:44.127460 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed51cd6-66b4-4fba-b301-7862648365bc None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:44.181020 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199309f0-9894-44ef-b3e8-1f1db412e940 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:44.216446 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391be07-09bd-452c-aa19-2db10d5c5db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:44.217156 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391be07-09bd-452c-aa19-2db10d5c5db4 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:44.217448 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391be07-09bd-452c-aa19-2db10d5c5db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:44.217732 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391be07-09bd-452c-aa19-2db10d5c5db4 None admin] PATH_INFO: `/v3/projects/4b2abb3ea36a4542819443c464ad5a0d` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:44.218988 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391be07-09bd-452c-aa19-2db10d5c5db4 None admin] RBAC: Authorizing `identity:get_project(project_id=4b2abb3ea36a4542819443c464ad5a0d)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:44.220457 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391be07-09bd-452c-aa19-2db10d5c5db4 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:44.222859 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 63/134] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:53:43 2022] GET /identity/v3/projects/4b2abb3ea36a4542819443c464ad5a0d => generated 312 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:44.223467 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391be07-09bd-452c-aa19-2db10d5c5db4 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:44.226173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:44.256960 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 72/135] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 3 04:53:43 2022] PUT /identity/v3/projects/1952481417bf4a2cad0b08d5a7676b61/users/b2cf0210955141068b832fc376bdc5ab/roles/561445e71aff40a0af7b01c539d3cfd9 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 04:53:44.257830 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed51cd6-66b4-4fba-b301-7862648365bc None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:44.385338 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391be07-09bd-452c-aa19-2db10d5c5db4 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:44.424290 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4314cee-0ff2-4645-8bb0-87ee4c0a3c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:44.425657 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4314cee-0ff2-4645-8bb0-87ee4c0a3c22 None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:44.425954 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4314cee-0ff2-4645-8bb0-87ee4c0a3c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:44.426244 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4314cee-0ff2-4645-8bb0-87ee4c0a3c22 None admin] PATH_INFO: `/v3/projects/1952481417bf4a2cad0b08d5a7676b61/users/d53b9368b2734205a2bdcd0bc2796161/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:44.428546 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4314cee-0ff2-4645-8bb0-87ee4c0a3c22 None admin] RBAC: Authorizing `identity:create_grant(project_id=1952481417bf4a2cad0b08d5a7676b61, user_id=d53b9368b2734205a2bdcd0bc2796161, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:44.430038 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4314cee-0ff2-4645-8bb0-87ee4c0a3c22 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:44.464908 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:44.483459 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed51cd6-66b4-4fba-b301-7862648365bc None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:44.498596 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 64/136] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 3 04:53:44 2022] PUT /identity/v3/projects/1952481417bf4a2cad0b08d5a7676b61/users/d53b9368b2734205a2bdcd0bc2796161/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 04:53:44.536207 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f92cc-b286-4e0f-b4ed-a0388331a2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:44.536900 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f92cc-b286-4e0f-b4ed-a0388331a2af None admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:44.537175 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f92cc-b286-4e0f-b4ed-a0388331a2af None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:44.537448 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f92cc-b286-4e0f-b4ed-a0388331a2af None admin] PATH_INFO: `/v3/projects/4b2abb3ea36a4542819443c464ad5a0d/users/b2cf0210955141068b832fc376bdc5ab/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:44.539426 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364f92cc-b286-4e0f-b4ed-a0388331a2af None admin] RBAC: Authorizing `identity:create_grant(project_id=4b2abb3ea36a4542819443c464ad5a0d, user_id=b2cf0210955141068b832fc376bdc5ab, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:44.540904 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364f92cc-b286-4e0f-b4ed-a0388331a2af None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:44.623228 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527aa7dc-c9c7-466a-855c-1e4aa378c0d4 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:44.623547 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527aa7dc-c9c7-466a-855c-1e4aa378c0d4 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:44.623881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527aa7dc-c9c7-466a-855c-1e4aa378c0d4 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:44.625330 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 65/137] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:53:44.632845 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a47294-9b8e-4421-b119-927180b4c8b3 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:44.633156 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a47294-9b8e-4421-b119-927180b4c8b3 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:44.633480 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a47294-9b8e-4421-b119-927180b4c8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:44.634546 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 73/138] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 3 04:53:44 2022] PUT /identity/v3/projects/4b2abb3ea36a4542819443c464ad5a0d/users/b2cf0210955141068b832fc376bdc5ab/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 04:53:44.655444 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-d0a47294-9b8e-4421-b119-927180b4c8b3 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:44.656769 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d0a47294-9b8e-4421-b119-927180b4c8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:44.737823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:44.752010 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 66/139] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:53:44.778788 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f92cc-b286-4e0f-b4ed-a0388331a2af None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:44.900328 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f92cc-b286-4e0f-b4ed-a0388331a2af None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:44.940790 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b9513-17be-456c-9703-111b4299e115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:44.941535 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b9513-17be-456c-9703-111b4299e115 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:44.941826 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b9513-17be-456c-9703-111b4299e115 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:44.942158 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b9513-17be-456c-9703-111b4299e115 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:44.944106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318b9513-17be-456c-9703-111b4299e115 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:44.945634 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318b9513-17be-456c-9703-111b4299e115 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:44.947556 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 74/140] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:53:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:44.953145 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a47294-9b8e-4421-b119-927180b4c8b3 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:45.071529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a47294-9b8e-4421-b119-927180b4c8b3 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:45.111952 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcf032-57be-405e-ad6a-dd13671a123b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:45.112637 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcf032-57be-405e-ad6a-dd13671a123b None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:45.112924 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcf032-57be-405e-ad6a-dd13671a123b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:45.113205 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcf032-57be-405e-ad6a-dd13671a123b None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:45.113920 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fcf032-57be-405e-ad6a-dd13671a123b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:45.115546 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fcf032-57be-405e-ad6a-dd13671a123b None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:45.234390 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 67/141] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 3 04:53:44 2022] POST /identity/v3/users => generated 283 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:53:45.390750 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb1118-8eb9-4509-9727-f4a63b4c8007 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:45.391473 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb1118-8eb9-4509-9727-f4a63b4c8007 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:45.391979 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb1118-8eb9-4509-9727-f4a63b4c8007 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:45.393615 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 75/142] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:45.399598 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea491857-2763-452b-a36b-f6286825f20d None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:45.400175 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea491857-2763-452b-a36b-f6286825f20d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:45.400674 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea491857-2763-452b-a36b-f6286825f20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:45.404885 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044d96a-79cc-435b-9051-ea189aea02c6 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:45.405440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044d96a-79cc-435b-9051-ea189aea02c6 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:45.406014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044d96a-79cc-435b-9051-ea189aea02c6 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:45.407923 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 76/143] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:45.418205 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0641b-0df7-4199-a7ea-d206fe6e3000 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:45.418809 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0641b-0df7-4199-a7ea-d206fe6e3000 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:45.419322 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0641b-0df7-4199-a7ea-d206fe6e3000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:45.449455 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-00f0641b-0df7-4199-a7ea-d206fe6e3000 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:45.451052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-00f0641b-0df7-4199-a7ea-d206fe6e3000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:45.451348 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-ea491857-2763-452b-a36b-f6286825f20d None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:45.452735 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ea491857-2763-452b-a36b-f6286825f20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:45.479297 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:45.541820 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:45.565580 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 77/144] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:53:45.606057 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 68/145] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:53:45.618244 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0641b-0df7-4199-a7ea-d206fe6e3000 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:45.633127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea491857-2763-452b-a36b-f6286825f20d None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:45.723362 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0641b-0df7-4199-a7ea-d206fe6e3000 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:45.759404 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f6b90-d988-4bd5-bd75-449d4e6f704b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:45.760059 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f6b90-d988-4bd5-bd75-449d4e6f704b None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:45.760329 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f6b90-d988-4bd5-bd75-449d4e6f704b None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:45.760602 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f6b90-d988-4bd5-bd75-449d4e6f704b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:45.768831 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea491857-2763-452b-a36b-f6286825f20d None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:45.774178 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f6b90-d988-4bd5-bd75-449d4e6f704b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:53:45.776122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-fb8f6b90-d988-4bd5-bd75-449d4e6f704b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 04:53:45.776122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:53:45.776122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:53:45.776122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:53:45.776122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:53:45.776122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:53:45.776122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:53:45.776122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:53:45.776122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:53:45.776122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:53:45.776122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:53:45.776122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:53:45.776122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:53:45.776122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:53:45.776122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:53:45.776122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:53:45.777240 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:53:45.777240 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:53:45.777240 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:53:45.777240 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:53:45.777240 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:53:45.777240 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:53:45.777240 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:53:45.777240 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:53:45.777240 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:53:45.777240 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:53:45.777240 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:53:45.777240 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:53:45.777240 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:53:45.777240 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:53:45.777240 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:53:45.777240 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:53:45.778000 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:53:45.778000 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:53:45.778000 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:53:45.778000 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:53:45.778000 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:53:45.778000 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:53:45.778000 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:53:45.778000 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:53:45.778000 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:53:45.778000 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:53:45.778000 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 04:53:45.778000 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:53:45.778000 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 78/146] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:53:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:53:45.784579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f6b90-d988-4bd5-bd75-449d4e6f704b None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:45.797451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:45.853558 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79d13c-b9bb-4524-9a1b-f1bb5f7a7430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:45.854593 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79d13c-b9bb-4524-9a1b-f1bb5f7a7430 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:45.855184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79d13c-b9bb-4524-9a1b-f1bb5f7a7430 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:45.855686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79d13c-b9bb-4524-9a1b-f1bb5f7a7430 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:45.883586 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79d13c-b9bb-4524-9a1b-f1bb5f7a7430 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:53:45.889300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-3c79d13c-b9bb-4524-9a1b-f1bb5f7a7430 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:53:45.889300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:53:45.889300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:53:45.889300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:53:45.889300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:53:45.889300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:53:45.889300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:53:45.889300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:53:45.889300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:53:45.889300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:53:45.889300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:53:45.889300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:53:45.889300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:53:45.889300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:53:45.889300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:53:45.889300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:53:45.890144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:53:45.890144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:53:45.890144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:53:45.890144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:53:45.890144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:53:45.890144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:53:45.890144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:53:45.890144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:53:45.890144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:53:45.890144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:53:45.890144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:53:45.890144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:53:45.890144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:53:45.890144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:53:45.890144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:53:45.890144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:53:45.890946 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:53:45.890946 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:53:45.890946 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:53:45.890946 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:53:45.890946 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:53:45.890946 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:53:45.890946 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:53:45.890946 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:53:45.890946 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:53:45.890946 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:53:45.890946 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:53:45.890946 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:53:45.894002 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f6b90-d988-4bd5-bd75-449d4e6f704b None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:45.894635 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 69/147] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:53:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:53:45.900460 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79d13c-b9bb-4524-9a1b-f1bb5f7a7430 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:45.932665 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5d178-f973-4fe1-8f07-a68d03d55b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:45.933212 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5d178-f973-4fe1-8f07-a68d03d55b44 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:45.933496 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5d178-f973-4fe1-8f07-a68d03d55b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:45.933768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5d178-f973-4fe1-8f07-a68d03d55b44 None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:45.936262 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5d178-f973-4fe1-8f07-a68d03d55b44 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:45.936262 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5d178-f973-4fe1-8f07-a68d03d55b44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:45.936262 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5d178-f973-4fe1-8f07-a68d03d55b44 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:45.945249 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 79/148] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 3 04:53:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:45.950758 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5d178-f973-4fe1-8f07-a68d03d55b44 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:45.954397 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:46.024176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:46.053630 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79d13c-b9bb-4524-9a1b-f1bb5f7a7430 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:46.056000 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5d178-f973-4fe1-8f07-a68d03d55b44 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:46.095623 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ec3c6-294c-42b8-8c5f-98924dab1ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:46.096272 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ec3c6-294c-42b8-8c5f-98924dab1ded None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:46.097111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ec3c6-294c-42b8-8c5f-98924dab1ded None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:46.097387 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ec3c6-294c-42b8-8c5f-98924dab1ded None admin] PATH_INFO: `/v3/users/4037c80bf46a404da45ebfd0596d5a12` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:46.098568 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ec3c6-294c-42b8-8c5f-98924dab1ded None admin] RBAC: Authorizing `identity:get_user(user_id=4037c80bf46a404da45ebfd0596d5a12)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:46.100176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ec3c6-294c-42b8-8c5f-98924dab1ded None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:46.101795 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 80/149] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:53:45 2022] GET /identity/v3/users/4037c80bf46a404da45ebfd0596d5a12 => generated 289 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:53:46.107518 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ec3c6-294c-42b8-8c5f-98924dab1ded None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:46.117036 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c7b5b-b425-4be2-8773-74ee478eabc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:46.117675 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c7b5b-b425-4be2-8773-74ee478eabc9 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:46.117967 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c7b5b-b425-4be2-8773-74ee478eabc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:46.118252 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c7b5b-b425-4be2-8773-74ee478eabc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:46.119030 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c7b5b-b425-4be2-8773-74ee478eabc9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:46.119345 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c7b5b-b425-4be2-8773-74ee478eabc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:46.120756 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c7b5b-b425-4be2-8773-74ee478eabc9 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:46.134766 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 70/150] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 3 04:53:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:46.140776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c7b5b-b425-4be2-8773-74ee478eabc9 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:46.220524 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ec3c6-294c-42b8-8c5f-98924dab1ded None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:46.256975 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89660b0-9232-4646-8c33-03e10873ec32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:46.257688 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89660b0-9232-4646-8c33-03e10873ec32 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:46.257969 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89660b0-9232-4646-8c33-03e10873ec32 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:46.258277 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89660b0-9232-4646-8c33-03e10873ec32 None admin] PATH_INFO: `/v3/projects/1952481417bf4a2cad0b08d5a7676b61` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:46.259492 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89660b0-9232-4646-8c33-03e10873ec32 None admin] RBAC: Authorizing `identity:get_project(project_id=1952481417bf4a2cad0b08d5a7676b61)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:46.260963 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89660b0-9232-4646-8c33-03e10873ec32 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:46.263011 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 81/151] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:53:46 2022] GET /identity/v3/projects/1952481417bf4a2cad0b08d5a7676b61 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:46.269064 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89660b0-9232-4646-8c33-03e10873ec32 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:46.285307 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c7b5b-b425-4be2-8773-74ee478eabc9 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:46.292674 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:46.293879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:46.344388 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d99e4-b0c7-43b8-aa71-b9b651aba896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:46.345045 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d99e4-b0c7-43b8-aa71-b9b651aba896 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:46.345338 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d99e4-b0c7-43b8-aa71-b9b651aba896 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:46.345626 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d99e4-b0c7-43b8-aa71-b9b651aba896 None admin] PATH_INFO: `/v3/users/b2cf0210955141068b832fc376bdc5ab` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:46.351606 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d99e4-b0c7-43b8-aa71-b9b651aba896 None admin] RBAC: Authorizing `identity:get_user(user_id=b2cf0210955141068b832fc376bdc5ab)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:46.353110 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d99e4-b0c7-43b8-aa71-b9b651aba896 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:46.355079 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 71/152] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:53:46 2022] GET /identity/v3/users/b2cf0210955141068b832fc376bdc5ab => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:53:46.360610 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d99e4-b0c7-43b8-aa71-b9b651aba896 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:46.384752 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89660b0-9232-4646-8c33-03e10873ec32 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:46.422123 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c40790-0970-42d2-b1ca-8eba818c5e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:46.422871 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c40790-0970-42d2-b1ca-8eba818c5e7a None admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:46.423165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c40790-0970-42d2-b1ca-8eba818c5e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:46.423437 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c40790-0970-42d2-b1ca-8eba818c5e7a None admin] PATH_INFO: `/v3/projects/1952481417bf4a2cad0b08d5a7676b61/users/4037c80bf46a404da45ebfd0596d5a12/roles/d7a7654a12dd4bd38542203bde537922` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:46.425467 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c40790-0970-42d2-b1ca-8eba818c5e7a None admin] RBAC: Authorizing `identity:create_grant(project_id=1952481417bf4a2cad0b08d5a7676b61, user_id=4037c80bf46a404da45ebfd0596d5a12, role_id=d7a7654a12dd4bd38542203bde537922)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:46.427051 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c40790-0970-42d2-b1ca-8eba818c5e7a None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:46.496435 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 82/153] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 3 04:53:46 2022] PUT /identity/v3/projects/1952481417bf4a2cad0b08d5a7676b61/users/4037c80bf46a404da45ebfd0596d5a12/roles/d7a7654a12dd4bd38542203bde537922 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 04:53:46.517289 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d99e4-b0c7-43b8-aa71-b9b651aba896 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:46.563301 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110110d-9915-4d8b-8abe-1754a66f8ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:46.563990 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110110d-9915-4d8b-8abe-1754a66f8ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:46.564274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110110d-9915-4d8b-8abe-1754a66f8ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:46.564550 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110110d-9915-4d8b-8abe-1754a66f8ac8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:46.610246 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:46.654510 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110110d-9915-4d8b-8abe-1754a66f8ac8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:46.656130 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110110d-9915-4d8b-8abe-1754a66f8ac8 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:46.791612 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 72/154] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:53:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 03 04:53:46.800987 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c40790-0970-42d2-b1ca-8eba818c5e7a None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:46.886269 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:46.922263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c40790-0970-42d2-b1ca-8eba818c5e7a None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:46.959957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d125f-7044-4683-a934-47b9361bfe8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:46.963652 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d125f-7044-4683-a934-47b9361bfe8d None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:46.963935 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d125f-7044-4683-a934-47b9361bfe8d None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:46.964215 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d125f-7044-4683-a934-47b9361bfe8d None admin] PATH_INFO: `/v3/projects/1952481417bf4a2cad0b08d5a7676b61` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:46.965518 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d125f-7044-4683-a934-47b9361bfe8d None admin] RBAC: Authorizing `identity:get_project(project_id=1952481417bf4a2cad0b08d5a7676b61)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:46.967031 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d125f-7044-4683-a934-47b9361bfe8d None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:46.969090 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 83/155] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:53:46 2022] GET /identity/v3/projects/1952481417bf4a2cad0b08d5a7676b61 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:46.975170 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110110d-9915-4d8b-8abe-1754a66f8ac8 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:47.083389 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:47.129966 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110110d-9915-4d8b-8abe-1754a66f8ac8 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:47.180653 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db685401-99c3-4362-9cb5-e8cf308b13cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:47.181281 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db685401-99c3-4362-9cb5-e8cf308b13cf None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:47.181564 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db685401-99c3-4362-9cb5-e8cf308b13cf None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:47.181842 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db685401-99c3-4362-9cb5-e8cf308b13cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:47.182638 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db685401-99c3-4362-9cb5-e8cf308b13cf None admin] RBAC: Adding query filter params (role.id=b0e0084bb31348a582616b1421f3b10c, scope.project.id=1952481417bf4a2cad0b08d5a7676b61, user.id=b2cf0210955141068b832fc376bdc5ab) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:47.182973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db685401-99c3-4362-9cb5-e8cf308b13cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:47.184432 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db685401-99c3-4362-9cb5-e8cf308b13cf None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:47.195757 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 73/156] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:53:46 2022] GET /identity/v3/role_assignments?user.id=b2cf0210955141068b832fc376bdc5ab&scope.project.id=1952481417bf4a2cad0b08d5a7676b61&role.id=b0e0084bb31348a582616b1421f3b10c => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:47.708885 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ad3a9-2bc9-49d8-9931-7cdd494ae899 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:47.709204 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ad3a9-2bc9-49d8-9931-7cdd494ae899 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:47.709447 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ad3a9-2bc9-49d8-9931-7cdd494ae899 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:47.710986 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 84/157] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:47.716216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b156a8-a836-4117-9fba-5981daa95796 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:47.716512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b156a8-a836-4117-9fba-5981daa95796 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:47.716800 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b156a8-a836-4117-9fba-5981daa95796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:47.743846 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-f0b156a8-a836-4117-9fba-5981daa95796 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:47.745062 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f0b156a8-a836-4117-9fba-5981daa95796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:47.837516 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 74/158] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:53:47.875516 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923ad3a9-2bc9-49d8-9931-7cdd494ae899 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:48.038499 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923ad3a9-2bc9-49d8-9931-7cdd494ae899 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:48.087644 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e6608-d713-4a68-bab5-55432446a441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:48.088299 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e6608-d713-4a68-bab5-55432446a441 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:48.088579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e6608-d713-4a68-bab5-55432446a441 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:48.088866 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e6608-d713-4a68-bab5-55432446a441 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:48.093447 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:48.107456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42e6608-d713-4a68-bab5-55432446a441 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:53:48.115612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-d42e6608-d713-4a68-bab5-55432446a441 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 04:53:48.115612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:53:48.115612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:53:48.115612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:53:48.115612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:53:48.115612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:53:48.115612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:53:48.115612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:53:48.115612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:53:48.115612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:53:48.115612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:53:48.115612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:53:48.115612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:53:48.115612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:53:48.115612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:53:48.115612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:53:48.116526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:53:48.116526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:53:48.116526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:53:48.116526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:53:48.116526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:53:48.116526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:53:48.116526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:53:48.116526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:53:48.116526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:53:48.116526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:53:48.116526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:53:48.116526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:53:48.116526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:53:48.116526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:53:48.116526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:53:48.116526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:53:48.117300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:53:48.117300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:53:48.117300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:53:48.117300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:53:48.117300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:53:48.117300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:53:48.117300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:53:48.117300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:53:48.117300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:53:48.117300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:53:48.117300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 04:53:48.117300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:53:48.117300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 85/159] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 3 04:53:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:53:48.122878 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b156a8-a836-4117-9fba-5981daa95796 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:48.227212 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:48.248062 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9c080-0553-4f1c-9dfd-f24f6afc7e51 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:48.248364 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9c080-0553-4f1c-9dfd-f24f6afc7e51 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:48.248621 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9c080-0553-4f1c-9dfd-f24f6afc7e51 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:48.250132 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 86/160] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:53:48.255263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1c2ea-b15c-43e4-93e3-98a650028f03 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:48.255548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1c2ea-b15c-43e4-93e3-98a650028f03 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:48.255818 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1c2ea-b15c-43e4-93e3-98a650028f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:48.271100 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b156a8-a836-4117-9fba-5981daa95796 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:48.293479 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-fef1c2ea-b15c-43e4-93e3-98a650028f03 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:48.294604 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-fef1c2ea-b15c-43e4-93e3-98a650028f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:48.320836 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cad3c5-cecd-4984-8c3e-5b8a72c05645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:48.321470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cad3c5-cecd-4984-8c3e-5b8a72c05645 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:48.321755 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cad3c5-cecd-4984-8c3e-5b8a72c05645 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:48.322032 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cad3c5-cecd-4984-8c3e-5b8a72c05645 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:48.322755 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cad3c5-cecd-4984-8c3e-5b8a72c05645 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:48.323088 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cad3c5-cecd-4984-8c3e-5b8a72c05645 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:48.324529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cad3c5-cecd-4984-8c3e-5b8a72c05645 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:48.337733 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 75/161] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:53:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:48.342955 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cad3c5-cecd-4984-8c3e-5b8a72c05645 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:48.415349 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 87/162] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 04:53:48.456972 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1c2ea-b15c-43e4-93e3-98a650028f03 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:48.487243 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cad3c5-cecd-4984-8c3e-5b8a72c05645 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:48.525541 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:48.548236 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8a20a-87ab-4749-a1ac-8bf68a3b781f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:48.548832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8a20a-87ab-4749-a1ac-8bf68a3b781f None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:48.549119 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8a20a-87ab-4749-a1ac-8bf68a3b781f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:48.549419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8a20a-87ab-4749-a1ac-8bf68a3b781f None admin] PATH_INFO: `/v3/users/b2cf0210955141068b832fc376bdc5ab` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:48.555590 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8a20a-87ab-4749-a1ac-8bf68a3b781f None admin] RBAC: Authorizing `identity:get_user(user_id=b2cf0210955141068b832fc376bdc5ab)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:48.557102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8a20a-87ab-4749-a1ac-8bf68a3b781f None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:48.559184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 76/163] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:53:48 2022] GET /identity/v3/users/b2cf0210955141068b832fc376bdc5ab => generated 275 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:48.565717 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8a20a-87ab-4749-a1ac-8bf68a3b781f None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:48.697170 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1c2ea-b15c-43e4-93e3-98a650028f03 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:48.734714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8a20a-87ab-4749-a1ac-8bf68a3b781f None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:48.747938 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:48.761985 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead813c6-94bd-4641-abcc-98a5531a6a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:48.767133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead813c6-94bd-4641-abcc-98a5531a6a90 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:48.767133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead813c6-94bd-4641-abcc-98a5531a6a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:48.767133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead813c6-94bd-4641-abcc-98a5531a6a90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:48.785785 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead813c6-94bd-4641-abcc-98a5531a6a90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:53:48.790820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-ead813c6-94bd-4641-abcc-98a5531a6a90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:53:48.790820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:53:48.790820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:53:48.790820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:53:48.790820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:53:48.790820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:53:48.790820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:53:48.790820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:53:48.790820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:53:48.790820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:53:48.790820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:53:48.790820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:53:48.790820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:53:48.790820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:53:48.790820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:53:48.790820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:53:48.791694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:53:48.791694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:53:48.791694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:53:48.791694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:53:48.791694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:53:48.791694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:53:48.791694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:53:48.791694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:53:48.791694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:53:48.791694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:53:48.791694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:53:48.791694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:53:48.791694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:53:48.791694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:53:48.791694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:53:48.791694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:53:48.792589 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:53:48.792589 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:53:48.792589 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:53:48.792589 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:53:48.792589 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:53:48.792589 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:53:48.792589 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:53:48.792589 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:53:48.792589 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:53:48.792589 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:53:48.792589 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:53:48.792589 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:53:48.792589 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 88/164] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:53:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:53:48.794128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdd776-8fb1-4065-b7c5-c9ac75124dfc None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:48.794128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdd776-8fb1-4065-b7c5-c9ac75124dfc None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:48.794301 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdd776-8fb1-4065-b7c5-c9ac75124dfc None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:48.808235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 89/165] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:48 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:48.808864 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bdd776-8fb1-4065-b7c5-c9ac75124dfc None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:48.810740 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa7689-0869-4eba-aec8-1300c50c74af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:48.811394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa7689-0869-4eba-aec8-1300c50c74af None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:48.811679 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa7689-0869-4eba-aec8-1300c50c74af None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:48.811955 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa7689-0869-4eba-aec8-1300c50c74af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:48.916617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:48.935444 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aa7689-0869-4eba-aec8-1300c50c74af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:48.940262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aa7689-0869-4eba-aec8-1300c50c74af None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:48.985542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bdd776-8fb1-4065-b7c5-c9ac75124dfc None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:49.072745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd1a72-b161-4542-8eed-dcb42e3e9970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:49.073886 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd1a72-b161-4542-8eed-dcb42e3e9970 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:49.075156 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd1a72-b161-4542-8eed-dcb42e3e9970 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:49.076307 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd1a72-b161-4542-8eed-dcb42e3e9970 None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:49.077452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd1a72-b161-4542-8eed-dcb42e3e9970 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:49.082992 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd1a72-b161-4542-8eed-dcb42e3e9970 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:49.084895 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd1a72-b161-4542-8eed-dcb42e3e9970 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:49.108536 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 90/166] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:53:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:49.112279 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c7379-bc5d-45fc-90e5-30e9afe18475 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:49.112773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c7379-bc5d-45fc-90e5-30e9afe18475 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:49.113250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c7379-bc5d-45fc-90e5-30e9afe18475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:49.139956 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-564c7379-bc5d-45fc-90e5-30e9afe18475 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:49.142634 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-564c7379-bc5d-45fc-90e5-30e9afe18475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:49.173438 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 77/167] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:53:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 03 04:53:49.174956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa7689-0869-4eba-aec8-1300c50c74af None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:49.247442 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:49.296921 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 91/168] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:53:49.297796 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c7379-bc5d-45fc-90e5-30e9afe18475 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:49.370651 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:49.392998 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa7689-0869-4eba-aec8-1300c50c74af None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:49.472298 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f23ba-4dad-48a9-8b91-95478bb37204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:49.473327 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f23ba-4dad-48a9-8b91-95478bb37204 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:49.473851 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f23ba-4dad-48a9-8b91-95478bb37204 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:49.474322 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f23ba-4dad-48a9-8b91-95478bb37204 None admin] PATH_INFO: `/v3/users/d53b9368b2734205a2bdcd0bc2796161` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:49.479829 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f23ba-4dad-48a9-8b91-95478bb37204 None admin] RBAC: Authorizing `identity:get_user(user_id=d53b9368b2734205a2bdcd0bc2796161)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:49.482138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f23ba-4dad-48a9-8b91-95478bb37204 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:49.484042 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 78/169] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:53:49 2022] GET /identity/v3/users/d53b9368b2734205a2bdcd0bc2796161 => generated 247 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:49.485016 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f23ba-4dad-48a9-8b91-95478bb37204 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:49.486762 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c7379-bc5d-45fc-90e5-30e9afe18475 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:49.560547 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31e713-1c53-4441-bbcd-02c9c0e18f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:49.561227 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31e713-1c53-4441-bbcd-02c9c0e18f7e None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:49.561579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31e713-1c53-4441-bbcd-02c9c0e18f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:49.561855 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31e713-1c53-4441-bbcd-02c9c0e18f7e None admin] PATH_INFO: `/v3/projects/1952481417bf4a2cad0b08d5a7676b61` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:49.563055 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31e713-1c53-4441-bbcd-02c9c0e18f7e None admin] RBAC: Authorizing `identity:get_project(project_id=1952481417bf4a2cad0b08d5a7676b61)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:49.564530 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31e713-1c53-4441-bbcd-02c9c0e18f7e None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:49.566752 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 92/170] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:53:49 2022] GET /identity/v3/projects/1952481417bf4a2cad0b08d5a7676b61 => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:49.567543 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31e713-1c53-4441-bbcd-02c9c0e18f7e None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:49.687358 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:49.694542 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f23ba-4dad-48a9-8b91-95478bb37204 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:49.732893 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c7a8d-9e10-4844-879d-1a661c45a273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:49.733874 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c7a8d-9e10-4844-879d-1a661c45a273 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:49.734173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c7a8d-9e10-4844-879d-1a661c45a273 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:49.734459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c7a8d-9e10-4844-879d-1a661c45a273 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:49.748019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c7a8d-9e10-4844-879d-1a661c45a273 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:53:49.754707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-c67c7a8d-9e10-4844-879d-1a661c45a273 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:53:49.754707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:53:49.754707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:53:49.754707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:53:49.754707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:53:49.754707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:53:49.754707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:53:49.754707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:53:49.754707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:53:49.754707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:53:49.754707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:53:49.754707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:53:49.754707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:53:49.754707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:53:49.754707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:53:49.754707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:53:49.755599 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:53:49.755599 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:53:49.755599 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:53:49.755599 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:53:49.755599 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:53:49.755599 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:53:49.755599 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:53:49.755599 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:53:49.755599 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:53:49.755599 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:53:49.755599 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:53:49.755599 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:53:49.755599 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:53:49.755599 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:53:49.755599 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:53:49.755599 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:53:49.756387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:53:49.756387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:53:49.756387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:53:49.756387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:53:49.756387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:53:49.756387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:53:49.756387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:53:49.756387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:53:49.756387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:53:49.756387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:53:49.756387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:53:49.756387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:53:49.756387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 79/171] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:53:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:53:49.757030 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c7a8d-9e10-4844-879d-1a661c45a273 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:49.796961 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:49.811783 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31e713-1c53-4441-bbcd-02c9c0e18f7e None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:49.878888 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c6bcc9-7a0d-47c8-8dd0-4d1de004a69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:49.879510 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6bcc9-7a0d-47c8-8dd0-4d1de004a69b None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:49.879786 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6bcc9-7a0d-47c8-8dd0-4d1de004a69b None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:49.880052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6bcc9-7a0d-47c8-8dd0-4d1de004a69b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:49.932242 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c7a8d-9e10-4844-879d-1a661c45a273 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:49.975342 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11356c18-366a-4d6c-b819-5ce91ffad1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:49.976139 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11356c18-366a-4d6c-b819-5ce91ffad1ac None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:49.976446 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11356c18-366a-4d6c-b819-5ce91ffad1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:49.976824 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11356c18-366a-4d6c-b819-5ce91ffad1ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:49.977594 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c6bcc9-7a0d-47c8-8dd0-4d1de004a69b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:49.977703 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11356c18-366a-4d6c-b819-5ce91ffad1ac None admin] RBAC: Adding query filter params (role.id=561445e71aff40a0af7b01c539d3cfd9, scope.project.id=1952481417bf4a2cad0b08d5a7676b61, user.id=b2cf0210955141068b832fc376bdc5ab) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:49.977912 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11356c18-366a-4d6c-b819-5ce91ffad1ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:49.979433 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11356c18-366a-4d6c-b819-5ce91ffad1ac None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:49.979806 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c6bcc9-7a0d-47c8-8dd0-4d1de004a69b None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:49.989830 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 80/172] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:53:49 2022] GET /identity/v3/role_assignments?user.id=b2cf0210955141068b832fc376bdc5ab&scope.project.id=1952481417bf4a2cad0b08d5a7676b61&role.id=561445e71aff40a0af7b01c539d3cfd9 => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:49.991560 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4c243-10de-4cdb-9e29-87dfbaec0b6c None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:49.992092 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4c243-10de-4cdb-9e29-87dfbaec0b6c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:49.992677 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4c243-10de-4cdb-9e29-87dfbaec0b6c None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:49.994422 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 81/173] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:49.995689 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4c243-10de-4cdb-9e29-87dfbaec0b6c None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:50.038266 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:50.097109 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 93/174] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:53:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:53:50.098717 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416088fa-0648-4235-96bb-db42abf57018 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:50.099028 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416088fa-0648-4235-96bb-db42abf57018 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:50.099304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416088fa-0648-4235-96bb-db42abf57018 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:50.100581 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 94/175] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:50.100952 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:50.102015 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edda29e-a43a-4de0-b4c8-c82c6a9f66f7 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:50.102297 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edda29e-a43a-4de0-b4c8-c82c6a9f66f7 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:50.106387 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edda29e-a43a-4de0-b4c8-c82c6a9f66f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:50.126145 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4c243-10de-4cdb-9e29-87dfbaec0b6c None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:50.126648 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-8edda29e-a43a-4de0-b4c8-c82c6a9f66f7 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:50.127936 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-8edda29e-a43a-4de0-b4c8-c82c6a9f66f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:50.169701 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c97ed2-9857-4a89-aa8a-b6177bb16d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:50.170814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c97ed2-9857-4a89-aa8a-b6177bb16d3a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:50.171378 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c97ed2-9857-4a89-aa8a-b6177bb16d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:50.171881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c97ed2-9857-4a89-aa8a-b6177bb16d3a None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:50.172823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c97ed2-9857-4a89-aa8a-b6177bb16d3a None admin] RBAC: Adding query filter params (name=member) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:50.173338 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c97ed2-9857-4a89-aa8a-b6177bb16d3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:50.175572 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c97ed2-9857-4a89-aa8a-b6177bb16d3a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:50.188947 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 82/176] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 3 04:53:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:50.190274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c97ed2-9857-4a89-aa8a-b6177bb16d3a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:50.207069 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 95/177] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:53:50.208374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d14e14-0faf-4ec2-bc79-7ab993229f26 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:50.208660 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d14e14-0faf-4ec2-bc79-7ab993229f26 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:50.208989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d14e14-0faf-4ec2-bc79-7ab993229f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:50.227823 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-02d14e14-0faf-4ec2-bc79-7ab993229f26 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:50.228939 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-02d14e14-0faf-4ec2-bc79-7ab993229f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:50.310889 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 96/178] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:53:50.311073 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d5d76-ea35-4d50-8057-a244e91a1889 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:50.311286 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d5d76-ea35-4d50-8057-a244e91a1889 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:50.311557 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d5d76-ea35-4d50-8057-a244e91a1889 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:50.312941 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 97/179] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:50.313585 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d5d76-ea35-4d50-8057-a244e91a1889 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:50.323256 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:50.334133 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c97ed2-9857-4a89-aa8a-b6177bb16d3a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:50.383053 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d9c43-7228-4997-8381-ea073e7d7835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:50.384317 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d9c43-7228-4997-8381-ea073e7d7835 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:50.384429 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d9c43-7228-4997-8381-ea073e7d7835 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:50.384647 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d9c43-7228-4997-8381-ea073e7d7835 None admin] PATH_INFO: `/v3/projects/1952481417bf4a2cad0b08d5a7676b61` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:50.386911 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d9c43-7228-4997-8381-ea073e7d7835 None admin] RBAC: Authorizing `identity:get_project(project_id=1952481417bf4a2cad0b08d5a7676b61)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:50.388559 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d9c43-7228-4997-8381-ea073e7d7835 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:50.390557 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 83/180] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:53:50 2022] GET /identity/v3/projects/1952481417bf4a2cad0b08d5a7676b61 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:50.391803 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d9c43-7228-4997-8381-ea073e7d7835 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:50.409190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:50.436339 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d5d76-ea35-4d50-8057-a244e91a1889 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:50.472117 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e35cd-f46a-4bb8-9cf3-edff8e495c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:50.472684 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e35cd-f46a-4bb8-9cf3-edff8e495c16 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:50.472956 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e35cd-f46a-4bb8-9cf3-edff8e495c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:50.473226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e35cd-f46a-4bb8-9cf3-edff8e495c16 None admin] PATH_INFO: `/v3/users/b2cf0210955141068b832fc376bdc5ab` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:50.474385 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e35cd-f46a-4bb8-9cf3-edff8e495c16 None admin] RBAC: Authorizing `identity:get_user(user_id=b2cf0210955141068b832fc376bdc5ab)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:50.475877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e35cd-f46a-4bb8-9cf3-edff8e495c16 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:50.477770 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 98/181] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:53:50 2022] GET /identity/v3/users/b2cf0210955141068b832fc376bdc5ab => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:50.484171 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886db533-76d7-4a98-baf8-5756a3e7a57f None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:50.484984 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886db533-76d7-4a98-baf8-5756a3e7a57f None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:50.484984 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886db533-76d7-4a98-baf8-5756a3e7a57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:50.499229 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d9c43-7228-4997-8381-ea073e7d7835 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:50.502580 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-886db533-76d7-4a98-baf8-5756a3e7a57f None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:50.503855 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-886db533-76d7-4a98-baf8-5756a3e7a57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:50.532850 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0474b605-0e09-4040-9a66-94c955ceaac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:50.533524 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474b605-0e09-4040-9a66-94c955ceaac7 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:50.534365 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474b605-0e09-4040-9a66-94c955ceaac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:50.534645 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474b605-0e09-4040-9a66-94c955ceaac7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:50.544843 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0474b605-0e09-4040-9a66-94c955ceaac7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:53:50.550190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-0474b605-0e09-4040-9a66-94c955ceaac7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 04:53:50.550190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:53:50.550190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:53:50.550190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:53:50.550190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:53:50.550190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:53:50.550190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:53:50.550190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:53:50.550190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:53:50.550190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:53:50.550190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:53:50.550190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:53:50.550190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:53:50.550190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:53:50.550190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:53:50.550190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:53:50.551122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:53:50.551122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:53:50.551122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:53:50.551122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:53:50.551122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:53:50.551122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:53:50.551122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:53:50.551122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:53:50.551122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:53:50.551122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:53:50.551122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:53:50.551122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:53:50.551122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:53:50.551122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:53:50.551122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:53:50.551122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:53:50.557291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:53:50.557291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:53:50.557291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:53:50.557291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:53:50.557291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:53:50.557291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:53:50.557291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:53:50.557291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:53:50.557291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:53:50.557291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:53:50.557291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 04:53:50.557291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:53:50.557291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 84/182] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 3 04:53:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:53:50.557291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0474b605-0e09-4040-9a66-94c955ceaac7 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:50.570651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 99/183] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:53:50.579796 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886db533-76d7-4a98-baf8-5756a3e7a57f None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:50.583790 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:50.631739 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:50.658140 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0474b605-0e09-4040-9a66-94c955ceaac7 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:50.685596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886db533-76d7-4a98-baf8-5756a3e7a57f None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:50.695029 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d2c2fd-9869-402e-9667-724de49cbe18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:50.695669 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2c2fd-9869-402e-9667-724de49cbe18 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:50.695959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2c2fd-9869-402e-9667-724de49cbe18 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:50.696241 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2c2fd-9869-402e-9667-724de49cbe18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:50.706797 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d2c2fd-9869-402e-9667-724de49cbe18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:53:50.712216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-f5d2c2fd-9869-402e-9667-724de49cbe18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:53:50.712216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:53:50.712216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:53:50.712216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:53:50.712216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:53:50.712216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:53:50.712216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:53:50.712216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:53:50.712216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:53:50.712216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:53:50.712216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:53:50.712216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:53:50.712216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:53:50.712216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:53:50.712216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:53:50.712216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:53:50.713102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:53:50.713102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:53:50.713102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:53:50.713102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:53:50.713102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:53:50.713102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:53:50.713102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:53:50.713102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:53:50.713102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:53:50.713102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:53:50.713102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:53:50.713102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:53:50.713102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:53:50.713102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:53:50.713102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:53:50.713102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:53:50.713899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:53:50.713899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:53:50.713899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:53:50.713899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:53:50.713899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:53:50.713899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:53:50.713899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:53:50.713899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:53:50.713899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:53:50.713899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:53:50.713899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:53:50.713899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:53:50.713899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 85/184] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:53:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:53:50.715079 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2970e4-b1df-43a0-9238-ce5c860cf4bb None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:50.715185 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2970e4-b1df-43a0-9238-ce5c860cf4bb None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:50.715471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2970e4-b1df-43a0-9238-ce5c860cf4bb None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:50.716854 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 86/185] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:50.717465 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2970e4-b1df-43a0-9238-ce5c860cf4bb None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:50.724601 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5caaf4-5a06-4a8f-9458-f01d6eba9699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:50.725164 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5caaf4-5a06-4a8f-9458-f01d6eba9699 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:50.725447 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5caaf4-5a06-4a8f-9458-f01d6eba9699 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:50.725723 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5caaf4-5a06-4a8f-9458-f01d6eba9699 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:50.726547 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5caaf4-5a06-4a8f-9458-f01d6eba9699 None admin] RBAC: Adding query filter params (role.id=e39d4ecea4c64d5794e1079c9be3ad36, scope.project.id=1952481417bf4a2cad0b08d5a7676b61, user.id=d53b9368b2734205a2bdcd0bc2796161) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:50.726885 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5caaf4-5a06-4a8f-9458-f01d6eba9699 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:50.728345 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5caaf4-5a06-4a8f-9458-f01d6eba9699 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:50.734209 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 100/186] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:53:50 2022] GET /identity/v3/role_assignments?user.id=d53b9368b2734205a2bdcd0bc2796161&scope.project.id=1952481417bf4a2cad0b08d5a7676b61&role.id=e39d4ecea4c64d5794e1079c9be3ad36 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:50.735024 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5caaf4-5a06-4a8f-9458-f01d6eba9699 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:50.817165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:50.839087 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2970e4-b1df-43a0-9238-ce5c860cf4bb None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:50.855929 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5caaf4-5a06-4a8f-9458-f01d6eba9699 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:50.879076 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46167c1-c1cb-404b-be58-eec82b358f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:50.879715 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46167c1-c1cb-404b-be58-eec82b358f12 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:50.880159 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46167c1-c1cb-404b-be58-eec82b358f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:50.880559 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46167c1-c1cb-404b-be58-eec82b358f12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:50.884476 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:50.898446 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe7db6-c53a-4f71-aeb8-e1307f5b39f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:50.900907 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe7db6-c53a-4f71-aeb8-e1307f5b39f6 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:50.900907 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe7db6-c53a-4f71-aeb8-e1307f5b39f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:50.900907 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe7db6-c53a-4f71-aeb8-e1307f5b39f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:50.900907 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe7db6-c53a-4f71-aeb8-e1307f5b39f6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:50.900907 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe7db6-c53a-4f71-aeb8-e1307f5b39f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:50.904531 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe7db6-c53a-4f71-aeb8-e1307f5b39f6 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:50.911350 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 101/187] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:53:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:50.912099 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe7db6-c53a-4f71-aeb8-e1307f5b39f6 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:50.972713 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46167c1-c1cb-404b-be58-eec82b358f12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:50.974207 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46167c1-c1cb-404b-be58-eec82b358f12 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:51.021221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe7db6-c53a-4f71-aeb8-e1307f5b39f6 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:51.049820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:51.055451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af00dd-ee92-45f7-9bf8-41f95bc6c11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:51.056174 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af00dd-ee92-45f7-9bf8-41f95bc6c11e None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:51.056457 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af00dd-ee92-45f7-9bf8-41f95bc6c11e None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:51.056781 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af00dd-ee92-45f7-9bf8-41f95bc6c11e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:51.067797 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af00dd-ee92-45f7-9bf8-41f95bc6c11e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:53:51.072733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-f9af00dd-ee92-45f7-9bf8-41f95bc6c11e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 04:53:51.072733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:53:51.072733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:53:51.072733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:53:51.072733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:53:51.072733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:53:51.072733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:53:51.072733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:53:51.072733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:53:51.072733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:53:51.072733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:53:51.072733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:53:51.072733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:53:51.072733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:53:51.072733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:53:51.072733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:53:51.073596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:53:51.073596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:53:51.073596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:53:51.073596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:53:51.073596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:53:51.073596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:53:51.073596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:53:51.073596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:53:51.073596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:53:51.073596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:53:51.073596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:53:51.073596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:53:51.073596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:53:51.073596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:53:51.073596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:53:51.073596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:53:51.077098 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:53:51.077098 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:53:51.077098 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:53:51.077098 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:53:51.077098 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:53:51.077098 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:53:51.077098 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:53:51.077098 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:53:51.077098 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:53:51.077098 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:53:51.077098 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 04:53:51.077098 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:53:51.077098 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 102/188] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:53:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:53:51.077098 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af00dd-ee92-45f7-9bf8-41f95bc6c11e None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:51.077881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 87/189] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:53:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:53:51.077881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333b855-fa88-41de-a71c-c2e7e5f2a54c None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:51.077881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333b855-fa88-41de-a71c-c2e7e5f2a54c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:51.077881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333b855-fa88-41de-a71c-c2e7e5f2a54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:51.095145 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-7333b855-fa88-41de-a71c-c2e7e5f2a54c None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:51.096365 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7333b855-fa88-41de-a71c-c2e7e5f2a54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:51.121860 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:51.161547 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 88/190] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:53:51.162450 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333b855-fa88-41de-a71c-c2e7e5f2a54c None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:51.184788 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af00dd-ee92-45f7-9bf8-41f95bc6c11e None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:51.218957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0a3b5-5488-4b1a-b2d1-b0b78846e1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:51.219581 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0a3b5-5488-4b1a-b2d1-b0b78846e1b0 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:51.219864 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0a3b5-5488-4b1a-b2d1-b0b78846e1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:51.220149 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0a3b5-5488-4b1a-b2d1-b0b78846e1b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:51.220873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c0a3b5-5488-4b1a-b2d1-b0b78846e1b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:51.221206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c0a3b5-5488-4b1a-b2d1-b0b78846e1b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:51.222613 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c0a3b5-5488-4b1a-b2d1-b0b78846e1b0 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:51.234056 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 103/191] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:53:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:51.234867 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0a3b5-5488-4b1a-b2d1-b0b78846e1b0 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:51.268690 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333b855-fa88-41de-a71c-c2e7e5f2a54c None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:51.293227 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:51.302907 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6e72d-694c-4ce8-8250-ba788ff27af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:51.303553 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6e72d-694c-4ce8-8250-ba788ff27af4 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:51.303838 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6e72d-694c-4ce8-8250-ba788ff27af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:51.304116 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6e72d-694c-4ce8-8250-ba788ff27af4 None admin] PATH_INFO: `/v3/users/4e13832df24c4448b1ca5a26869875f7` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:51.318156 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6e72d-694c-4ce8-8250-ba788ff27af4 None admin] RBAC: Authorizing `identity:get_user(user_id=4e13832df24c4448b1ca5a26869875f7)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:51.319662 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6e72d-694c-4ce8-8250-ba788ff27af4 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:51.321338 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 89/192] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:53:51 2022] GET /identity/v3/users/4e13832df24c4448b1ca5a26869875f7 => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:51.322103 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6e72d-694c-4ce8-8250-ba788ff27af4 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:51.345339 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0a3b5-5488-4b1a-b2d1-b0b78846e1b0 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:51.345531 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:51.385894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0546ce3-2e72-4d99-9b7d-a2f2f35ff40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:51.386521 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0546ce3-2e72-4d99-9b7d-a2f2f35ff40f None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:51.386805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0546ce3-2e72-4d99-9b7d-a2f2f35ff40f None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:51.387121 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0546ce3-2e72-4d99-9b7d-a2f2f35ff40f None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:51.387832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0546ce3-2e72-4d99-9b7d-a2f2f35ff40f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:51.388143 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0546ce3-2e72-4d99-9b7d-a2f2f35ff40f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:51.389548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0546ce3-2e72-4d99-9b7d-a2f2f35ff40f None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:51.397623 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 104/193] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 3 04:53:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:51.398416 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0546ce3-2e72-4d99-9b7d-a2f2f35ff40f None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:51.426760 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6e72d-694c-4ce8-8250-ba788ff27af4 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:51.460681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5336de69-6596-4ced-ab4c-84d7bb82635e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:51.461339 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336de69-6596-4ced-ab4c-84d7bb82635e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:51.461658 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336de69-6596-4ced-ab4c-84d7bb82635e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:51.462000 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336de69-6596-4ced-ab4c-84d7bb82635e None admin] PATH_INFO: `/v3/projects/4b2abb3ea36a4542819443c464ad5a0d` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:51.463261 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5336de69-6596-4ced-ab4c-84d7bb82635e None admin] RBAC: Authorizing `identity:get_project(project_id=4b2abb3ea36a4542819443c464ad5a0d)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:51.464738 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5336de69-6596-4ced-ab4c-84d7bb82635e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:51.466433 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 90/194] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:53:51 2022] GET /identity/v3/projects/4b2abb3ea36a4542819443c464ad5a0d => generated 312 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:51.468302 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5336de69-6596-4ced-ab4c-84d7bb82635e None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:51.500355 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0546ce3-2e72-4d99-9b7d-a2f2f35ff40f None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:51.515402 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:51.537837 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b379362-ea94-430f-b7fc-fc3d34203e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:51.538478 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b379362-ea94-430f-b7fc-fc3d34203e42 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:51.538756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b379362-ea94-430f-b7fc-fc3d34203e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:51.539057 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b379362-ea94-430f-b7fc-fc3d34203e42 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:51.540363 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b379362-ea94-430f-b7fc-fc3d34203e42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:51.542063 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b379362-ea94-430f-b7fc-fc3d34203e42 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:51.543743 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 105/195] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:53:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:51.544442 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b379362-ea94-430f-b7fc-fc3d34203e42 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:51.574437 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:51.584005 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5336de69-6596-4ced-ab4c-84d7bb82635e None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:51.622769 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c50e2f-b1aa-433c-8617-c4d0cf2e019b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:51.623475 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c50e2f-b1aa-433c-8617-c4d0cf2e019b None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:51.623757 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c50e2f-b1aa-433c-8617-c4d0cf2e019b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:51.624028 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c50e2f-b1aa-433c-8617-c4d0cf2e019b None admin] PATH_INFO: `/v3/users/4e13832df24c4448b1ca5a26869875f7` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:51.625252 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c50e2f-b1aa-433c-8617-c4d0cf2e019b None admin] RBAC: Authorizing `identity:get_user(user_id=4e13832df24c4448b1ca5a26869875f7)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:51.626869 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c50e2f-b1aa-433c-8617-c4d0cf2e019b None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:51.628507 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 91/196] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:53:51 2022] GET /identity/v3/users/4e13832df24c4448b1ca5a26869875f7 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:51.629312 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c50e2f-b1aa-433c-8617-c4d0cf2e019b None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:51.650328 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b379362-ea94-430f-b7fc-fc3d34203e42 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:51.685817 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a1423-cdce-460c-b8d6-2019d9c9453e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:51.686531 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a1423-cdce-460c-b8d6-2019d9c9453e None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:51.686904 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a1423-cdce-460c-b8d6-2019d9c9453e None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:51.687182 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a1423-cdce-460c-b8d6-2019d9c9453e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:51.734946 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:51.736630 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c50e2f-b1aa-433c-8617-c4d0cf2e019b None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:51.756340 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8a1423-cdce-460c-b8d6-2019d9c9453e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:51.757831 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8a1423-cdce-460c-b8d6-2019d9c9453e None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:51.771019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed026b4a-2201-44e1-b14d-2f37f261590a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:51.771687 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed026b4a-2201-44e1-b14d-2f37f261590a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:51.771969 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed026b4a-2201-44e1-b14d-2f37f261590a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:51.772245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed026b4a-2201-44e1-b14d-2f37f261590a None admin] PATH_INFO: `/v3/users/4037c80bf46a404da45ebfd0596d5a12` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:51.773414 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed026b4a-2201-44e1-b14d-2f37f261590a None admin] RBAC: Authorizing `identity:get_user(user_id=4037c80bf46a404da45ebfd0596d5a12)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:51.774869 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed026b4a-2201-44e1-b14d-2f37f261590a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:51.776744 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 92/197] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:53:51 2022] GET /identity/v3/users/4037c80bf46a404da45ebfd0596d5a12 => generated 289 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:51.777512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed026b4a-2201-44e1-b14d-2f37f261590a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:51.805774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:51.868377 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 106/198] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:53:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:53:51.869219 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a1423-cdce-460c-b8d6-2019d9c9453e None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:51.883395 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed026b4a-2201-44e1-b14d-2f37f261590a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:51.917030 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec466bca-9a67-4646-a47f-fa44869178f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:51.917649 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec466bca-9a67-4646-a47f-fa44869178f0 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:51.917932 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec466bca-9a67-4646-a47f-fa44869178f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:51.918207 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec466bca-9a67-4646-a47f-fa44869178f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:51.919025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec466bca-9a67-4646-a47f-fa44869178f0 None admin] RBAC: Adding query filter params (role.id=b0e0084bb31348a582616b1421f3b10c, scope.project.id=4b2abb3ea36a4542819443c464ad5a0d, user.id=b2cf0210955141068b832fc376bdc5ab) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:51.919335 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec466bca-9a67-4646-a47f-fa44869178f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:51.920782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec466bca-9a67-4646-a47f-fa44869178f0 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:51.927177 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 93/199] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:53:51 2022] GET /identity/v3/role_assignments?user.id=b2cf0210955141068b832fc376bdc5ab&scope.project.id=4b2abb3ea36a4542819443c464ad5a0d&role.id=b0e0084bb31348a582616b1421f3b10c => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:51.927970 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec466bca-9a67-4646-a47f-fa44869178f0 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:51.952724 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:51.974361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a1423-cdce-460c-b8d6-2019d9c9453e None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:52.012732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57df9f-b963-4fdf-92e5-dfa777e3b782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:52.013384 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57df9f-b963-4fdf-92e5-dfa777e3b782 None admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:52.013666 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57df9f-b963-4fdf-92e5-dfa777e3b782 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:52.013942 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57df9f-b963-4fdf-92e5-dfa777e3b782 None admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:52.014666 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57df9f-b963-4fdf-92e5-dfa777e3b782 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:52.016283 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57df9f-b963-4fdf-92e5-dfa777e3b782 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:52.049587 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:52.059924 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec466bca-9a67-4646-a47f-fa44869178f0 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:52.097986 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39807939-14c6-4022-be44-787db9d9f41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:52.098610 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39807939-14c6-4022-be44-787db9d9f41c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:52.098965 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39807939-14c6-4022-be44-787db9d9f41c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:52.099238 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39807939-14c6-4022-be44-787db9d9f41c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:52.126422 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 107/200] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 3 04:53:51 2022] POST /identity/v3/users => generated 297 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:53:52.134115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57df9f-b963-4fdf-92e5-dfa777e3b782 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:52.193572 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39807939-14c6-4022-be44-787db9d9f41c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:52.195104 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39807939-14c6-4022-be44-787db9d9f41c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:52.264343 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57df9f-b963-4fdf-92e5-dfa777e3b782 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:52.301528 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:52.302613 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b5722-451c-489f-b351-77bed266ce5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:52.303288 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b5722-451c-489f-b351-77bed266ce5e None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:52.303575 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b5722-451c-489f-b351-77bed266ce5e None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:52.303858 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b5722-451c-489f-b351-77bed266ce5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:52.313138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 94/201] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:53:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:53:52.314287 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39807939-14c6-4022-be44-787db9d9f41c None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:52.315350 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:52.388883 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7b5722-451c-489f-b351-77bed266ce5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:52.390450 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7b5722-451c-489f-b351-77bed266ce5e None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:52.444098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39807939-14c6-4022-be44-787db9d9f41c None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:52.485739 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c7d3c-5dad-46f7-98a3-4230b727a9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:52.486384 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c7d3c-5dad-46f7-98a3-4230b727a9cb None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:52.486689 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c7d3c-5dad-46f7-98a3-4230b727a9cb None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:52.487051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c7d3c-5dad-46f7-98a3-4230b727a9cb None admin] PATH_INFO: `/v3/projects/255c31b4828b499abeb900572c1ce254` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:52.488272 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c7d3c-5dad-46f7-98a3-4230b727a9cb None admin] RBAC: Authorizing `identity:get_project(project_id=255c31b4828b499abeb900572c1ce254)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:52.490427 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c7d3c-5dad-46f7-98a3-4230b727a9cb None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:52.492783 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 95/202] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:53:52 2022] GET /identity/v3/projects/255c31b4828b499abeb900572c1ce254 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:52.494653 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c7d3c-5dad-46f7-98a3-4230b727a9cb None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:52.527640 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 108/203] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:53:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:53:52.532389 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b5722-451c-489f-b351-77bed266ce5e None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:52.574815 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:52.577563 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:52.625171 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c7d3c-5dad-46f7-98a3-4230b727a9cb None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:52.650259 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b5722-451c-489f-b351-77bed266ce5e None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:52.673017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe9802-e2d9-44b2-aeaf-683f404f66d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:52.674015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe9802-e2d9-44b2-aeaf-683f404f66d8 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:52.675163 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe9802-e2d9-44b2-aeaf-683f404f66d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:52.675676 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe9802-e2d9-44b2-aeaf-683f404f66d8 None admin] PATH_INFO: `/v3/projects/255c31b4828b499abeb900572c1ce254` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:52.677075 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe9802-e2d9-44b2-aeaf-683f404f66d8 None admin] RBAC: Authorizing `identity:get_project(project_id=255c31b4828b499abeb900572c1ce254)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:52.678733 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe9802-e2d9-44b2-aeaf-683f404f66d8 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:52.681037 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 96/204] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:53:52 2022] GET /identity/v3/projects/255c31b4828b499abeb900572c1ce254 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:52.682160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe9802-e2d9-44b2-aeaf-683f404f66d8 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:52.690371 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a72b6-6069-4e9e-add1-80aa1fc84099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:52.691032 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a72b6-6069-4e9e-add1-80aa1fc84099 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:52.691316 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a72b6-6069-4e9e-add1-80aa1fc84099 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:52.691592 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a72b6-6069-4e9e-add1-80aa1fc84099 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:52.692388 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5a72b6-6069-4e9e-add1-80aa1fc84099 None admin] RBAC: Adding query filter params (role.id=561445e71aff40a0af7b01c539d3cfd9, scope.project.id=255c31b4828b499abeb900572c1ce254, user.id=4e13832df24c4448b1ca5a26869875f7) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:52.693091 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5a72b6-6069-4e9e-add1-80aa1fc84099 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:52.694285 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5a72b6-6069-4e9e-add1-80aa1fc84099 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:52.700859 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 109/205] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:53:52 2022] GET /identity/v3/role_assignments?user.id=4e13832df24c4448b1ca5a26869875f7&scope.project.id=255c31b4828b499abeb900572c1ce254&role.id=561445e71aff40a0af7b01c539d3cfd9 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:52.701641 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a72b6-6069-4e9e-add1-80aa1fc84099 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:52.795926 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe9802-e2d9-44b2-aeaf-683f404f66d8 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:52.819567 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a72b6-6069-4e9e-add1-80aa1fc84099 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:52.821127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:52.826090 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:52.835867 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5804d-4306-49b3-a874-acf315bb8c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:52.836977 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5804d-4306-49b3-a874-acf315bb8c6b None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:52.837470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5804d-4306-49b3-a874-acf315bb8c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:52.837980 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5804d-4306-49b3-a874-acf315bb8c6b None admin] PATH_INFO: `/v3/projects/1952481417bf4a2cad0b08d5a7676b61` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:52.839606 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5804d-4306-49b3-a874-acf315bb8c6b None admin] RBAC: Authorizing `identity:get_project(project_id=1952481417bf4a2cad0b08d5a7676b61)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:52.841516 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5804d-4306-49b3-a874-acf315bb8c6b None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:52.851323 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 97/206] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:53:52 2022] GET /identity/v3/projects/1952481417bf4a2cad0b08d5a7676b61 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:52.851323 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5804d-4306-49b3-a874-acf315bb8c6b None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:52.858471 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4167ec74-0482-49af-9b5f-f924aa182dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:52.859146 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167ec74-0482-49af-9b5f-f924aa182dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:52.859430 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167ec74-0482-49af-9b5f-f924aa182dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:52.859707 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167ec74-0482-49af-9b5f-f924aa182dd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:52.860520 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4167ec74-0482-49af-9b5f-f924aa182dd1 None admin] RBAC: Adding query filter params (role.id=e39d4ecea4c64d5794e1079c9be3ad36, scope.project.id=255c31b4828b499abeb900572c1ce254, user.id=4e13832df24c4448b1ca5a26869875f7) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:52.860838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4167ec74-0482-49af-9b5f-f924aa182dd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:52.862571 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4167ec74-0482-49af-9b5f-f924aa182dd1 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:52.868937 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 110/207] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:53:52 2022] GET /identity/v3/role_assignments?user.id=4e13832df24c4448b1ca5a26869875f7&scope.project.id=255c31b4828b499abeb900572c1ce254&role.id=e39d4ecea4c64d5794e1079c9be3ad36 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:52.976064 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5804d-4306-49b3-a874-acf315bb8c6b None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:53.017887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff99001-4875-4cb9-847b-3a379f3fb046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:53.018535 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff99001-4875-4cb9-847b-3a379f3fb046 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:53.018818 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff99001-4875-4cb9-847b-3a379f3fb046 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:53.019124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff99001-4875-4cb9-847b-3a379f3fb046 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:53.019931 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff99001-4875-4cb9-847b-3a379f3fb046 None admin] RBAC: Adding query filter params (role.id=d7a7654a12dd4bd38542203bde537922, scope.project.id=1952481417bf4a2cad0b08d5a7676b61, user.id=4037c80bf46a404da45ebfd0596d5a12) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:53.020240 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff99001-4875-4cb9-847b-3a379f3fb046 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:53.021728 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff99001-4875-4cb9-847b-3a379f3fb046 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:53.029494 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 98/208] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:53:52 2022] GET /identity/v3/role_assignments?user.id=4037c80bf46a404da45ebfd0596d5a12&scope.project.id=1952481417bf4a2cad0b08d5a7676b61&role.id=d7a7654a12dd4bd38542203bde537922 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:53:54.457030 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7143b-c83e-4691-8419-7638e75aa1fb None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:54.457567 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7143b-c83e-4691-8419-7638e75aa1fb None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:54.457567 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7143b-c83e-4691-8419-7638e75aa1fb None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:54.458973 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 111/209] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:54.463661 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c353e-e568-40b4-9db6-1aae11249dd6 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:54.464088 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c353e-e568-40b4-9db6-1aae11249dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:54.464180 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734bd79-fe0f-429b-8e1e-b4a47130a008 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:54.464260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c353e-e568-40b4-9db6-1aae11249dd6 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:54.464346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734bd79-fe0f-429b-8e1e-b4a47130a008 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:54.464544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734bd79-fe0f-429b-8e1e-b4a47130a008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:54.465561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 99/210] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:54.470359 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdf1aa-5dc2-43bc-a631-dfa1aab8e375 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:54.470653 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdf1aa-5dc2-43bc-a631-dfa1aab8e375 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:54.471048 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdf1aa-5dc2-43bc-a631-dfa1aab8e375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:54.484721 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-c734bd79-fe0f-429b-8e1e-b4a47130a008 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:54.486373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-c734bd79-fe0f-429b-8e1e-b4a47130a008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:54.489572 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-cfbdf1aa-5dc2-43bc-a631-dfa1aab8e375 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:54.490818 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-cfbdf1aa-5dc2-43bc-a631-dfa1aab8e375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:54.547694 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:54.562711 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 100/211] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:53:54.580489 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 112/212] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 04:53:54.588028 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbdf1aa-5dc2-43bc-a631-dfa1aab8e375 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:54.605535 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734bd79-fe0f-429b-8e1e-b4a47130a008 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:54.703291 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:54.717944 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734bd79-fe0f-429b-8e1e-b4a47130a008 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:54.718126 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbdf1aa-5dc2-43bc-a631-dfa1aab8e375 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:54.747796 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7c4be-769e-4e2c-960a-318b5b850ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:54.748795 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7c4be-769e-4e2c-960a-318b5b850ddf None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:54.749302 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7c4be-769e-4e2c-960a-318b5b850ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:54.749797 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7c4be-769e-4e2c-960a-318b5b850ddf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:54.751368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7c4be-769e-4e2c-960a-318b5b850ddf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:54.753206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7c4be-769e-4e2c-960a-318b5b850ddf None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:54.755076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 113/213] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:53:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:53:54.762992 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7c4be-769e-4e2c-960a-318b5b850ddf None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:54.764155 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb1eb4-c9c2-4d6a-a28e-1c2baff7ed1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:54.764817 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb1eb4-c9c2-4d6a-a28e-1c2baff7ed1a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:54.765103 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb1eb4-c9c2-4d6a-a28e-1c2baff7ed1a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:54.765390 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb1eb4-c9c2-4d6a-a28e-1c2baff7ed1a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:54.776591 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb1eb4-c9c2-4d6a-a28e-1c2baff7ed1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:53:54.781921 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-adbb1eb4-c9c2-4d6a-a28e-1c2baff7ed1a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:53:54.781921 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:53:54.781921 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:53:54.781921 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:53:54.781921 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:53:54.781921 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:53:54.781921 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:53:54.781921 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:53:54.781921 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:53:54.781921 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:53:54.781921 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:53:54.781921 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:53:54.781921 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:53:54.781921 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:53:54.781921 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:53:54.781921 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:53:54.782858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:53:54.782858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:53:54.782858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:53:54.782858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:53:54.782858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:53:54.782858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:53:54.782858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:53:54.782858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:53:54.782858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:53:54.782858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:53:54.782858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:53:54.782858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:53:54.782858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:53:54.782858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:53:54.782858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:53:54.782858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:53:54.784262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:53:54.784262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:53:54.784262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:53:54.784262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:53:54.784262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:53:54.784262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:53:54.784262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:53:54.784262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:53:54.784262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:53:54.784262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:53:54.784262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:53:54.784262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:53:54.784262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 101/214] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:53:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:53:54.794962 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb1eb4-c9c2-4d6a-a28e-1c2baff7ed1a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:54.878157 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7c4be-769e-4e2c-960a-318b5b850ddf None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:54.883324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:54.918377 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f6d2e-fe0c-4e19-b7ee-3cd0938886cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:54.919198 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f6d2e-fe0c-4e19-b7ee-3cd0938886cc None admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:54.919586 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f6d2e-fe0c-4e19-b7ee-3cd0938886cc None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:54.922219 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f6d2e-fe0c-4e19-b7ee-3cd0938886cc None admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:54.923150 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f6d2e-fe0c-4e19-b7ee-3cd0938886cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:54.925552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f6d2e-fe0c-4e19-b7ee-3cd0938886cc None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:54.931290 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb1eb4-c9c2-4d6a-a28e-1c2baff7ed1a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:54.972086 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c06fc9-b9df-4b48-96b3-0219090f24ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:54.972629 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c06fc9-b9df-4b48-96b3-0219090f24ce None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:54.972915 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c06fc9-b9df-4b48-96b3-0219090f24ce None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:54.973642 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c06fc9-b9df-4b48-96b3-0219090f24ce None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:54.974375 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c06fc9-b9df-4b48-96b3-0219090f24ce None admin] RBAC: Adding query filter params (name=member) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:54.974682 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c06fc9-b9df-4b48-96b3-0219090f24ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:54.976128 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c06fc9-b9df-4b48-96b3-0219090f24ce None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:54.984669 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 102/215] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 3 04:53:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:54.987176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae3a19-d2cc-49f6-b6c7-70e3cb3d03c1 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:54.987463 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae3a19-d2cc-49f6-b6c7-70e3cb3d03c1 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:54.987741 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae3a19-d2cc-49f6-b6c7-70e3cb3d03c1 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:54.989096 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 103/216] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:54.993470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae3a19-d2cc-49f6-b6c7-70e3cb3d03c1 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:55.046168 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 114/217] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 3 04:53:54 2022] POST /identity/v3/users => generated 297 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 04:53:55.051440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a5b0e-fe53-41fb-86a3-0159bc0386d6 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:55.051440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a5b0e-fe53-41fb-86a3-0159bc0386d6 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:55.051677 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a5b0e-fe53-41fb-86a3-0159bc0386d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:55.082860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-b05a5b0e-fe53-41fb-86a3-0159bc0386d6 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:55.084173 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-b05a5b0e-fe53-41fb-86a3-0159bc0386d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:55.103569 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:55.149337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae3a19-d2cc-49f6-b6c7-70e3cb3d03c1 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:55.169123 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:55.175052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 115/218] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:53:55.175052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9d93b-707a-47bf-b26f-5fa3a28ed591 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:55.175052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9d93b-707a-47bf-b26f-5fa3a28ed591 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:55.175395 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9d93b-707a-47bf-b26f-5fa3a28ed591 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:55.176978 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 116/219] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:55.182881 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a67976-6245-4da1-bbb9-cc951797ae13 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:55.183179 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a67976-6245-4da1-bbb9-cc951797ae13 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:55.183456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a67976-6245-4da1-bbb9-cc951797ae13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:55.202767 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-93a67976-6245-4da1-bbb9-cc951797ae13 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:55.204035 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-93a67976-6245-4da1-bbb9-cc951797ae13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:55.212575 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29851226-9333-476e-a655-304904cf34ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:55.213858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29851226-9333-476e-a655-304904cf34ee None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:55.214175 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29851226-9333-476e-a655-304904cf34ee None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:55.214435 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29851226-9333-476e-a655-304904cf34ee None admin] PATH_INFO: `/v3/users/fbabb1d409a7427f96084ad22c80b8ac` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:55.237483 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29851226-9333-476e-a655-304904cf34ee None admin] RBAC: Authorizing `identity:get_user(user_id=fbabb1d409a7427f96084ad22c80b8ac)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:55.239216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29851226-9333-476e-a655-304904cf34ee None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:55.240963 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 104/220] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:53:54 2022] GET /identity/v3/users/fbabb1d409a7427f96084ad22c80b8ac => generated 297 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:55.241742 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29851226-9333-476e-a655-304904cf34ee None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:55.293123 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 117/221] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 04:53:55.299121 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a67976-6245-4da1-bbb9-cc951797ae13 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:55.372748 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29851226-9333-476e-a655-304904cf34ee None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:55.405034 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:55.434486 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd864b-6df4-4389-8843-ac1c6508bc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:55.435186 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd864b-6df4-4389-8843-ac1c6508bc1d None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:55.435468 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd864b-6df4-4389-8843-ac1c6508bc1d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:55.435739 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd864b-6df4-4389-8843-ac1c6508bc1d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:55.445404 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a67976-6245-4da1-bbb9-cc951797ae13 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:55.449437 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fd864b-6df4-4389-8843-ac1c6508bc1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:53:55.454508 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-15fd864b-6df4-4389-8843-ac1c6508bc1d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 04:53:55.454508 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:53:55.454508 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:53:55.454508 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:53:55.454508 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:53:55.454508 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:53:55.454508 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:53:55.454508 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:53:55.454508 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:53:55.454508 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:53:55.454508 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:53:55.454508 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:53:55.454508 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:53:55.454508 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:53:55.454508 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:53:55.454508 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:53:55.469614 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:53:55.469614 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:53:55.469614 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:53:55.469614 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:53:55.469614 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:53:55.469614 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:53:55.469614 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:53:55.469614 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:53:55.469614 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:53:55.469614 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:53:55.469614 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:53:55.469614 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:53:55.469614 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:53:55.469614 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:53:55.469614 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:53:55.469614 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:53:55.471513 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:53:55.471513 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:53:55.471513 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:53:55.471513 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:53:55.471513 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:53:55.471513 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:53:55.471513 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:53:55.471513 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:53:55.471513 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:53:55.471513 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:53:55.471513 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 04:53:55.471513 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:53:55.471513 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 105/222] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 3 04:53:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:53:55.471513 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd864b-6df4-4389-8843-ac1c6508bc1d None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:55.498333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d3380-713a-4798-9e09-aae4e8f2f50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:55.498333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d3380-713a-4798-9e09-aae4e8f2f50f None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:55.498333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d3380-713a-4798-9e09-aae4e8f2f50f None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:55.498333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d3380-713a-4798-9e09-aae4e8f2f50f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:55.499080 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:55.588401 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd864b-6df4-4389-8843-ac1c6508bc1d None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:55.600901 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606d3380-713a-4798-9e09-aae4e8f2f50f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:55.604016 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606d3380-713a-4798-9e09-aae4e8f2f50f None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:55.634973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473917e-004e-4060-a9c2-21e78aaa8a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:55.635645 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473917e-004e-4060-a9c2-21e78aaa8a84 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:55.635930 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473917e-004e-4060-a9c2-21e78aaa8a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:55.636646 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473917e-004e-4060-a9c2-21e78aaa8a84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:55.651731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473917e-004e-4060-a9c2-21e78aaa8a84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:53:55.657933 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-e473917e-004e-4060-a9c2-21e78aaa8a84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:53:55.657933 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:53:55.657933 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:53:55.657933 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:53:55.657933 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:53:55.657933 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:53:55.657933 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:53:55.657933 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:53:55.657933 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:53:55.657933 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:53:55.657933 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:53:55.657933 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:53:55.657933 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:53:55.657933 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:53:55.657933 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:53:55.657933 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:53:55.659636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:53:55.659636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:53:55.659636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:53:55.659636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:53:55.659636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:53:55.659636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:53:55.659636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:53:55.659636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:53:55.659636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:53:55.659636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:53:55.659636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:53:55.659636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:53:55.659636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:53:55.659636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:53:55.659636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:53:55.659636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:53:55.660420 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:53:55.660420 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:53:55.660420 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:53:55.660420 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:53:55.660420 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:53:55.660420 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:53:55.660420 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:53:55.660420 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:53:55.660420 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:53:55.660420 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:53:55.660420 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:53:55.660420 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:53:55.660420 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 106/223] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:53:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:53:55.661105 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473917e-004e-4060-a9c2-21e78aaa8a84 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:55.722330 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:55.748280 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 118/224] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:53:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:53:55.755019 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d3380-713a-4798-9e09-aae4e8f2f50f None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:55.787277 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:55.795739 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473917e-004e-4060-a9c2-21e78aaa8a84 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:55.834252 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8523183-7b5c-4d16-b76d-4765f38f351d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:55.835286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8523183-7b5c-4d16-b76d-4765f38f351d None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:55.835781 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8523183-7b5c-4d16-b76d-4765f38f351d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:55.836280 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8523183-7b5c-4d16-b76d-4765f38f351d None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:55.837199 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8523183-7b5c-4d16-b76d-4765f38f351d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:55.837706 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8523183-7b5c-4d16-b76d-4765f38f351d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:55.839344 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8523183-7b5c-4d16-b76d-4765f38f351d None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:55.852307 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 107/225] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:53:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:55.853649 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8523183-7b5c-4d16-b76d-4765f38f351d None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:55.889184 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d3380-713a-4798-9e09-aae4e8f2f50f None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:55.939655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334b746-2c46-47c1-8a43-76474ea163bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:55.940318 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334b746-2c46-47c1-8a43-76474ea163bc None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:55.940589 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334b746-2c46-47c1-8a43-76474ea163bc None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:55.940868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334b746-2c46-47c1-8a43-76474ea163bc None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:55.941640 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334b746-2c46-47c1-8a43-76474ea163bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:55.941941 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334b746-2c46-47c1-8a43-76474ea163bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:55.943913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334b746-2c46-47c1-8a43-76474ea163bc None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:55.960393 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 119/226] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:53:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:55.960393 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334b746-2c46-47c1-8a43-76474ea163bc None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:55.973877 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8523183-7b5c-4d16-b76d-4765f38f351d None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:56.010717 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:56.024672 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6a4c0-64aa-48ab-b224-90e1deff9bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:56.025745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6a4c0-64aa-48ab-b224-90e1deff9bfa None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:56.026415 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6a4c0-64aa-48ab-b224-90e1deff9bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:56.026629 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6a4c0-64aa-48ab-b224-90e1deff9bfa None admin] PATH_INFO: `/v3/projects/255c31b4828b499abeb900572c1ce254` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:56.028265 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e6a4c0-64aa-48ab-b224-90e1deff9bfa None admin] RBAC: Authorizing `identity:get_project(project_id=255c31b4828b499abeb900572c1ce254)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:56.030316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e6a4c0-64aa-48ab-b224-90e1deff9bfa None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:56.038005 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 108/227] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:53:55 2022] GET /identity/v3/projects/255c31b4828b499abeb900572c1ce254 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:56.038005 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6a4c0-64aa-48ab-b224-90e1deff9bfa None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:56.059478 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:56.080012 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334b746-2c46-47c1-8a43-76474ea163bc None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:56.117591 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc08a1-8d61-40a5-b74b-0dd1cc922c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:56.118606 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc08a1-8d61-40a5-b74b-0dd1cc922c3e None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:56.119162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc08a1-8d61-40a5-b74b-0dd1cc922c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:56.119670 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc08a1-8d61-40a5-b74b-0dd1cc922c3e None admin] PATH_INFO: `/v3/users/4e13832df24c4448b1ca5a26869875f7` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:56.121059 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dc08a1-8d61-40a5-b74b-0dd1cc922c3e None admin] RBAC: Authorizing `identity:get_user(user_id=4e13832df24c4448b1ca5a26869875f7)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:56.122871 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dc08a1-8d61-40a5-b74b-0dd1cc922c3e None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:56.124704 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 120/228] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:53:55 2022] GET /identity/v3/users/4e13832df24c4448b1ca5a26869875f7 => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:56.125844 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc08a1-8d61-40a5-b74b-0dd1cc922c3e None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:56.168578 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6a4c0-64aa-48ab-b224-90e1deff9bfa None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:56.217118 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6923b-44e5-4f26-8c07-887a3f1d0429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:56.217761 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6923b-44e5-4f26-8c07-887a3f1d0429 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:56.218046 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6923b-44e5-4f26-8c07-887a3f1d0429 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:56.218321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6923b-44e5-4f26-8c07-887a3f1d0429 None admin] PATH_INFO: `/v3/users/4e13832df24c4448b1ca5a26869875f7` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:56.220153 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc6923b-44e5-4f26-8c07-887a3f1d0429 None admin] RBAC: Authorizing `identity:get_user(user_id=4e13832df24c4448b1ca5a26869875f7)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:56.222157 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc6923b-44e5-4f26-8c07-887a3f1d0429 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:56.228436 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 109/229] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:53:56 2022] GET /identity/v3/users/4e13832df24c4448b1ca5a26869875f7 => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:56.230297 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6923b-44e5-4f26-8c07-887a3f1d0429 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:56.251645 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc08a1-8d61-40a5-b74b-0dd1cc922c3e None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:56.269027 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:56.300265 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6c648-d552-4930-9f95-2f1a956d5ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:56.301812 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6c648-d552-4930-9f95-2f1a956d5ceb None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:56.302312 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6c648-d552-4930-9f95-2f1a956d5ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:56.303067 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6c648-d552-4930-9f95-2f1a956d5ceb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:56.304312 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a6c648-d552-4930-9f95-2f1a956d5ceb None admin] RBAC: Adding query filter params (role.id=b0e0084bb31348a582616b1421f3b10c, scope.project.id=255c31b4828b499abeb900572c1ce254, user.id=fbabb1d409a7427f96084ad22c80b8ac) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:56.304860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a6c648-d552-4930-9f95-2f1a956d5ceb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:56.307620 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a6c648-d552-4930-9f95-2f1a956d5ceb None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:56.315523 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 121/230] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:53:56 2022] GET /identity/v3/role_assignments?user.id=fbabb1d409a7427f96084ad22c80b8ac&scope.project.id=255c31b4828b499abeb900572c1ce254&role.id=b0e0084bb31348a582616b1421f3b10c => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:56.316575 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6c648-d552-4930-9f95-2f1a956d5ceb None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:56.333709 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:56.363110 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6923b-44e5-4f26-8c07-887a3f1d0429 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:56.444302 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637e385-5e99-43d7-84c9-29a7f98d00a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:56.445032 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637e385-5e99-43d7-84c9-29a7f98d00a3 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:56.445324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637e385-5e99-43d7-84c9-29a7f98d00a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:56.445666 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637e385-5e99-43d7-84c9-29a7f98d00a3 None admin] PATH_INFO: `/v3/projects/255c31b4828b499abeb900572c1ce254` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:56.450021 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6c648-d552-4930-9f95-2f1a956d5ceb None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:56.452066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1637e385-5e99-43d7-84c9-29a7f98d00a3 None admin] RBAC: Authorizing `identity:get_project(project_id=255c31b4828b499abeb900572c1ce254)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:56.454364 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1637e385-5e99-43d7-84c9-29a7f98d00a3 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:56.455841 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 110/231] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:53:56 2022] GET /identity/v3/projects/255c31b4828b499abeb900572c1ce254 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:56.462147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637e385-5e99-43d7-84c9-29a7f98d00a3 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:56.499288 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6981d54-bf5f-4a7a-8d10-e2ded11cf514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:56.500950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6981d54-bf5f-4a7a-8d10-e2ded11cf514 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:56.501434 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6981d54-bf5f-4a7a-8d10-e2ded11cf514 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:56.501934 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6981d54-bf5f-4a7a-8d10-e2ded11cf514 None admin] PATH_INFO: `/v3/projects/255c31b4828b499abeb900572c1ce254` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:56.503463 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6981d54-bf5f-4a7a-8d10-e2ded11cf514 None admin] RBAC: Authorizing `identity:get_project(project_id=255c31b4828b499abeb900572c1ce254)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:56.505630 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6981d54-bf5f-4a7a-8d10-e2ded11cf514 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:56.508207 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 122/232] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:53:56 2022] GET /identity/v3/projects/255c31b4828b499abeb900572c1ce254 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:56.516875 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6981d54-bf5f-4a7a-8d10-e2ded11cf514 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:56.608292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:56.728490 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637e385-5e99-43d7-84c9-29a7f98d00a3 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:56.770565 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6981d54-bf5f-4a7a-8d10-e2ded11cf514 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:56.847105 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129fa831-af0b-45a9-9250-9029628c4481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:56.848275 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fa831-af0b-45a9-9250-9029628c4481 None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:56.848785 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fa831-af0b-45a9-9250-9029628c4481 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:56.849293 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fa831-af0b-45a9-9250-9029628c4481 None admin] PATH_INFO: `/v3/projects/255c31b4828b499abeb900572c1ce254/users/4e13832df24c4448b1ca5a26869875f7/roles/561445e71aff40a0af7b01c539d3cfd9` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:56.854034 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129fa831-af0b-45a9-9250-9029628c4481 None admin] RBAC: Authorizing `identity:create_grant(project_id=255c31b4828b499abeb900572c1ce254, user_id=4e13832df24c4448b1ca5a26869875f7, role_id=561445e71aff40a0af7b01c539d3cfd9)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:56.857067 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129fa831-af0b-45a9-9250-9029628c4481 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:56.875907 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d375c4-1ec4-4163-94b7-fdbaabd77f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:56.875907 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d375c4-1ec4-4163-94b7-fdbaabd77f98 None admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:56.875907 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d375c4-1ec4-4163-94b7-fdbaabd77f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:56.875907 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d375c4-1ec4-4163-94b7-fdbaabd77f98 None admin] PATH_INFO: `/v3/projects/255c31b4828b499abeb900572c1ce254/users/4e13832df24c4448b1ca5a26869875f7/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:56.876558 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d375c4-1ec4-4163-94b7-fdbaabd77f98 None admin] RBAC: Authorizing `identity:create_grant(project_id=255c31b4828b499abeb900572c1ce254, user_id=4e13832df24c4448b1ca5a26869875f7, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:56.878061 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d375c4-1ec4-4163-94b7-fdbaabd77f98 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:56.932126 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 111/233] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 3 04:53:56 2022] PUT /identity/v3/projects/255c31b4828b499abeb900572c1ce254/users/4e13832df24c4448b1ca5a26869875f7/roles/561445e71aff40a0af7b01c539d3cfd9 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 04:53:57.011419 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 123/234] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 3 04:53:56 2022] PUT /identity/v3/projects/255c31b4828b499abeb900572c1ce254/users/4e13832df24c4448b1ca5a26869875f7/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 04:53:57.560030 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c9691-a5b1-4bc4-b484-104a7c4e852a None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:57.560406 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c9691-a5b1-4bc4-b484-104a7c4e852a None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:57.560587 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c9691-a5b1-4bc4-b484-104a7c4e852a None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:57.562212 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 112/235] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:53:57.571120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5350148-6174-4e9c-8ce0-4a06229be395 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:57.571120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5350148-6174-4e9c-8ce0-4a06229be395 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:57.571120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5350148-6174-4e9c-8ce0-4a06229be395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:57.595895 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-c5350148-6174-4e9c-8ce0-4a06229be395 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:57.597485 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-c5350148-6174-4e9c-8ce0-4a06229be395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:57.690215 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 124/236] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:53:57.720106 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c9691-a5b1-4bc4-b484-104a7c4e852a None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:57.737816 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a681d4-11cb-4256-8891-c4ec044cb021 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:57.738152 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a681d4-11cb-4256-8891-c4ec044cb021 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:57.738385 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a681d4-11cb-4256-8891-c4ec044cb021 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:57.739945 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 125/237] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:53:57.743176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:57.745003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206f31c-8e3f-440e-9817-9f01d35dc91e None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:57.745297 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206f31c-8e3f-440e-9817-9f01d35dc91e None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:57.745577 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206f31c-8e3f-440e-9817-9f01d35dc91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:57.775478 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-a206f31c-8e3f-440e-9817-9f01d35dc91e None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:57.780050 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-a206f31c-8e3f-440e-9817-9f01d35dc91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:57.832465 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:57.902350 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 126/238] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 04:53:57.918141 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c9691-a5b1-4bc4-b484-104a7c4e852a None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:57.926452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206f31c-8e3f-440e-9817-9f01d35dc91e None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:57.973072 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66564504-57ed-44ca-b7ae-e0e5f6e40fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:57.974161 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66564504-57ed-44ca-b7ae-e0e5f6e40fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:57.975265 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66564504-57ed-44ca-b7ae-e0e5f6e40fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:57.976162 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66564504-57ed-44ca-b7ae-e0e5f6e40fc5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:57.997986 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66564504-57ed-44ca-b7ae-e0e5f6e40fc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:53:58.003897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-66564504-57ed-44ca-b7ae-e0e5f6e40fc5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 04:53:58.003897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:53:58.003897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:53:58.003897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:53:58.003897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:53:58.003897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:53:58.003897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:53:58.003897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:53:58.003897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:53:58.003897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:53:58.003897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:53:58.003897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:53:58.003897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:53:58.003897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:53:58.003897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:53:58.003897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:53:58.004776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:53:58.004776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:53:58.004776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:53:58.004776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:53:58.004776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:53:58.004776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:53:58.004776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:53:58.004776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:53:58.004776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:53:58.004776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:53:58.004776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:53:58.004776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:53:58.004776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:53:58.004776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:53:58.004776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:53:58.004776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:53:58.005581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:53:58.005581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:53:58.005581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:53:58.005581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:53:58.005581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:53:58.005581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:53:58.005581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:53:58.005581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:53:58.005581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:53:58.005581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:53:58.005581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 04:53:58.005581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:53:58.007623 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 113/239] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:53:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:53:58.013491 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66564504-57ed-44ca-b7ae-e0e5f6e40fc5 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:58.091665 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:58.117228 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66564504-57ed-44ca-b7ae-e0e5f6e40fc5 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:58.152719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a442137-910f-4c54-8a00-c06e7bce0410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:58.153580 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a442137-910f-4c54-8a00-c06e7bce0410 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:58.154107 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a442137-910f-4c54-8a00-c06e7bce0410 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:58.154550 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a442137-910f-4c54-8a00-c06e7bce0410 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:58.155441 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a442137-910f-4c54-8a00-c06e7bce0410 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:58.155955 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a442137-910f-4c54-8a00-c06e7bce0410 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:58.157599 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a442137-910f-4c54-8a00-c06e7bce0410 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:58.166805 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 114/240] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 3 04:53:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:53:58.179043 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a442137-910f-4c54-8a00-c06e7bce0410 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:58.222321 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206f31c-8e3f-440e-9817-9f01d35dc91e None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:58.292719 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ae6b9-b9ff-459c-88af-9124cdec525f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:58.293380 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ae6b9-b9ff-459c-88af-9124cdec525f None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:58.293660 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ae6b9-b9ff-459c-88af-9124cdec525f None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:58.294695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ae6b9-b9ff-459c-88af-9124cdec525f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:58.296538 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ae6b9-b9ff-459c-88af-9124cdec525f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:58.297999 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ae6b9-b9ff-459c-88af-9124cdec525f None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:58.300192 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 127/241] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:53:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:53:58.306804 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ae6b9-b9ff-459c-88af-9124cdec525f None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:58.356027 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:58.369277 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a442137-910f-4c54-8a00-c06e7bce0410 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:58.432950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ae6b9-b9ff-459c-88af-9124cdec525f None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:58.438075 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737a3bd-21f0-415f-ae6a-4b56ea400463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:58.438732 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737a3bd-21f0-415f-ae6a-4b56ea400463 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:58.439040 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737a3bd-21f0-415f-ae6a-4b56ea400463 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:58.439316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737a3bd-21f0-415f-ae6a-4b56ea400463 None admin] PATH_INFO: `/v3/users/e9de1de3860a40fbabbea9687378bcda` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:58.452994 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:58.460367 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4737a3bd-21f0-415f-ae6a-4b56ea400463 None admin] RBAC: Authorizing `identity:get_user(user_id=e9de1de3860a40fbabbea9687378bcda)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:58.461925 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4737a3bd-21f0-415f-ae6a-4b56ea400463 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:58.466294 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 115/242] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:53:58 2022] GET /identity/v3/users/e9de1de3860a40fbabbea9687378bcda => generated 297 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:58.472255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737a3bd-21f0-415f-ae6a-4b56ea400463 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:58.475248 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7c842-b25a-45d4-9d6b-4cb109a54752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:58.476159 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7c842-b25a-45d4-9d6b-4cb109a54752 None admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:58.476659 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7c842-b25a-45d4-9d6b-4cb109a54752 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:58.477226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7c842-b25a-45d4-9d6b-4cb109a54752 None admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:58.478150 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f7c842-b25a-45d4-9d6b-4cb109a54752 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:58.479841 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f7c842-b25a-45d4-9d6b-4cb109a54752 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:58.576487 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 128/243] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 3 04:53:58 2022] POST /identity/v3/users => generated 293 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:53:58.616483 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737a3bd-21f0-415f-ae6a-4b56ea400463 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:58.668062 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77443ff5-f830-48df-9a1e-5dab553920f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:58.668716 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77443ff5-f830-48df-9a1e-5dab553920f5 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:58.669000 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77443ff5-f830-48df-9a1e-5dab553920f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:58.669279 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77443ff5-f830-48df-9a1e-5dab553920f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:58.760641 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:58.771738 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77443ff5-f830-48df-9a1e-5dab553920f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:58.773223 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77443ff5-f830-48df-9a1e-5dab553920f5 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:58.882539 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 116/244] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:53:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:53:58.887768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7c842-b25a-45d4-9d6b-4cb109a54752 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:58.916058 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4bd7a-bc5a-4e62-928c-a3bb1cb4db81 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:58.916368 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4bd7a-bc5a-4e62-928c-a3bb1cb4db81 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:58.916626 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4bd7a-bc5a-4e62-928c-a3bb1cb4db81 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:58.918120 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 117/245] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:53:58.924441 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146cfe6-16a9-476b-9956-2180170b9f0d None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:58.924725 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146cfe6-16a9-476b-9956-2180170b9f0d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:58.925005 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146cfe6-16a9-476b-9956-2180170b9f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:58.946576 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-f146cfe6-16a9-476b-9956-2180170b9f0d None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:58.950287 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f146cfe6-16a9-476b-9956-2180170b9f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:58.991913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:59.012050 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7c842-b25a-45d4-9d6b-4cb109a54752 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:59.055507 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 118/246] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 04:53:59.059672 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00ce374-32f5-4497-9816-0d8fd7d31012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:59.060346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ce374-32f5-4497-9816-0d8fd7d31012 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:59.060689 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ce374-32f5-4497-9816-0d8fd7d31012 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:59.060978 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ce374-32f5-4497-9816-0d8fd7d31012 None admin] PATH_INFO: `/v3/projects/255c31b4828b499abeb900572c1ce254` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:59.062244 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00ce374-32f5-4497-9816-0d8fd7d31012 None admin] RBAC: Authorizing `identity:get_project(project_id=255c31b4828b499abeb900572c1ce254)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:59.063887 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00ce374-32f5-4497-9816-0d8fd7d31012 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:59.066179 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 129/247] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:53:58 2022] GET /identity/v3/projects/255c31b4828b499abeb900572c1ce254 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:53:59.072056 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f146cfe6-16a9-476b-9956-2180170b9f0d None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:59.086660 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00ce374-32f5-4497-9816-0d8fd7d31012 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:59.109980 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:59.201394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f146cfe6-16a9-476b-9956-2180170b9f0d None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:59.233301 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00ce374-32f5-4497-9816-0d8fd7d31012 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:59.278174 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a97d5-7180-437b-8021-8c1d99e129d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:59.278375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a97d5-7180-437b-8021-8c1d99e129d0 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:59.278611 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a97d5-7180-437b-8021-8c1d99e129d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:59.278816 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a97d5-7180-437b-8021-8c1d99e129d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:59.285201 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718c1cf-925b-411d-ba88-92f9abf9554e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:59.285854 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718c1cf-925b-411d-ba88-92f9abf9554e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:59.286108 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718c1cf-925b-411d-ba88-92f9abf9554e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:59.286385 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718c1cf-925b-411d-ba88-92f9abf9554e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:59.287272 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e718c1cf-925b-411d-ba88-92f9abf9554e None admin] RBAC: Adding query filter params (role.id=d7a7654a12dd4bd38542203bde537922, scope.project.id=255c31b4828b499abeb900572c1ce254, user.id=e9de1de3860a40fbabbea9687378bcda) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:59.287585 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e718c1cf-925b-411d-ba88-92f9abf9554e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:59.289052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e718c1cf-925b-411d-ba88-92f9abf9554e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:59.296762 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 119/248] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:53:59 2022] GET /identity/v3/role_assignments?user.id=e9de1de3860a40fbabbea9687378bcda&scope.project.id=255c31b4828b499abeb900572c1ce254&role.id=d7a7654a12dd4bd38542203bde537922 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:59.300206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a97d5-7180-437b-8021-8c1d99e129d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:53:59.307504 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-114a97d5-7180-437b-8021-8c1d99e129d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:53:59.307504 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:53:59.307504 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:53:59.307504 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:53:59.307504 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:53:59.307504 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:53:59.307504 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:53:59.307504 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:53:59.307504 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:53:59.307504 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:53:59.307504 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:53:59.307504 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:53:59.307504 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:53:59.307504 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:53:59.307504 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:53:59.307504 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:53:59.308533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:53:59.308533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:53:59.308533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:53:59.308533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:53:59.308533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:53:59.308533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:53:59.308533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:53:59.308533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:53:59.308533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:53:59.308533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:53:59.308533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:53:59.308533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:53:59.308533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:53:59.308533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:53:59.308533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:53:59.308533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:53:59.309339 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:53:59.309339 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:53:59.309339 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:53:59.309339 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:53:59.309339 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:53:59.309339 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:53:59.309339 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:53:59.309339 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:53:59.309339 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:53:59.309339 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:53:59.309339 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:53:59.309339 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:53:59.310677 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 130/249] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:53:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:53:59.317366 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718c1cf-925b-411d-ba88-92f9abf9554e None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:59.366824 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097c939-4e24-4e63-ad85-6822554642f8 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:59.367194 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097c939-4e24-4e63-ad85-6822554642f8 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:59.367446 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097c939-4e24-4e63-ad85-6822554642f8 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:59.368846 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 131/250] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:53:59.374170 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b22ef6-29d7-43ca-be61-06ba4cf911bb None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:59.374478 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b22ef6-29d7-43ca-be61-06ba4cf911bb None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:59.374726 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b22ef6-29d7-43ca-be61-06ba4cf911bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:59.411781 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-05b22ef6-29d7-43ca-be61-06ba4cf911bb None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:59.413355 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-05b22ef6-29d7-43ca-be61-06ba4cf911bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:53:59.438576 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:59.574299 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:59.613767 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718c1cf-925b-411d-ba88-92f9abf9554e None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:59.624070 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 132/251] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 04:53:59.682835 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b22ef6-29d7-43ca-be61-06ba4cf911bb None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:59.730048 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701d002-f866-4216-9a64-a8c9d0894f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:59.731207 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701d002-f866-4216-9a64-a8c9d0894f74 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:59.731742 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701d002-f866-4216-9a64-a8c9d0894f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:59.732229 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701d002-f866-4216-9a64-a8c9d0894f74 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:59.733109 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701d002-f866-4216-9a64-a8c9d0894f74 None admin] RBAC: Adding query filter params (name=member) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:53:59.733726 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701d002-f866-4216-9a64-a8c9d0894f74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:53:59.735319 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701d002-f866-4216-9a64-a8c9d0894f74 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:53:59.763820 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 120/252] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 3 04:53:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 447 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:53:59.773251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701d002-f866-4216-9a64-a8c9d0894f74 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:53:59.883475 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b22ef6-29d7-43ca-be61-06ba4cf911bb None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:53:59.925055 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7db30f-5e68-4590-b31f-23d2cab1f63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:53:59.930873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7db30f-5e68-4590-b31f-23d2cab1f63d None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:59.930873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7db30f-5e68-4590-b31f-23d2cab1f63d None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:59.930873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7db30f-5e68-4590-b31f-23d2cab1f63d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:59.946095 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7db30f-5e68-4590-b31f-23d2cab1f63d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:53:59.952137 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-9b7db30f-5e68-4590-b31f-23d2cab1f63d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 04:53:59.952137 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:53:59.952137 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:53:59.952137 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:53:59.952137 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:53:59.952137 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:53:59.952137 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:53:59.952137 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:53:59.952137 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:53:59.952137 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:53:59.952137 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:53:59.952137 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:53:59.952137 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:53:59.952137 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:53:59.952137 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:53:59.952137 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:53:59.953025 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:53:59.953025 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:53:59.953025 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:53:59.953025 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:53:59.953025 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:53:59.953025 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:53:59.953025 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:53:59.953025 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:53:59.953025 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:53:59.953025 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:53:59.953025 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:53:59.953025 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:53:59.953025 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:53:59.953025 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:53:59.953025 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:53:59.953025 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:53:59.953821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:53:59.953821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:53:59.953821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:53:59.953821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:53:59.953821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:53:59.953821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:53:59.953821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:53:59.953821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:53:59.953821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:53:59.953821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:53:59.953821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 04:53:59.953821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:53:59.955041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 133/253] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 3 04:53:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:53:59.955306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110c4b5-5702-46fa-9475-b723cb4c19a6 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:59.955592 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110c4b5-5702-46fa-9475-b723cb4c19a6 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:59.957583 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110c4b5-5702-46fa-9475-b723cb4c19a6 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:59.957583 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 134/254] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:53:59.962954 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a6e6f-87b0-4f56-8c3b-405d7d8cf9e4 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:53:59.963456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a6e6f-87b0-4f56-8c3b-405d7d8cf9e4 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:53:59.963881 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a6e6f-87b0-4f56-8c3b-405d7d8cf9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:53:59.977166 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:53:59.990998 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-d53a6e6f-87b0-4f56-8c3b-405d7d8cf9e4 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:53:59.992397 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-d53a6e6f-87b0-4f56-8c3b-405d7d8cf9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:00.077551 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 135/255] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:53:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 04:54:00.078422 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701d002-f866-4216-9a64-a8c9d0894f74 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:00.078873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a6e6f-87b0-4f56-8c3b-405d7d8cf9e4 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:00.168579 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf2a66-d4f1-439c-9a89-28f8df02ed66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:00.169312 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf2a66-d4f1-439c-9a89-28f8df02ed66 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:00.169806 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf2a66-d4f1-439c-9a89-28f8df02ed66 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:00.170079 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf2a66-d4f1-439c-9a89-28f8df02ed66 None admin] PATH_INFO: `/v3/users/fbabb1d409a7427f96084ad22c80b8ac` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:00.175659 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cf2a66-d4f1-439c-9a89-28f8df02ed66 None admin] RBAC: Authorizing `identity:get_user(user_id=fbabb1d409a7427f96084ad22c80b8ac)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:00.177308 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cf2a66-d4f1-439c-9a89-28f8df02ed66 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:00.179486 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 121/256] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:53:59 2022] GET /identity/v3/users/fbabb1d409a7427f96084ad22c80b8ac => generated 297 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:00.180366 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf2a66-d4f1-439c-9a89-28f8df02ed66 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:00.219419 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a6e6f-87b0-4f56-8c3b-405d7d8cf9e4 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:00.227984 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:00.256525 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23cd4a-ea86-405b-b04b-ba628938e3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:00.257421 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23cd4a-ea86-405b-b04b-ba628938e3bf None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:00.257874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23cd4a-ea86-405b-b04b-ba628938e3bf None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:00.258313 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23cd4a-ea86-405b-b04b-ba628938e3bf None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:00.259364 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a23cd4a-ea86-405b-b04b-ba628938e3bf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:00.259889 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a23cd4a-ea86-405b-b04b-ba628938e3bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:00.261483 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a23cd4a-ea86-405b-b04b-ba628938e3bf None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:00.263189 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:00.273297 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 136/257] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:54:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:00.274594 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23cd4a-ea86-405b-b04b-ba628938e3bf None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:00.411860 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf2a66-d4f1-439c-9a89-28f8df02ed66 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:00.428283 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23cd4a-ea86-405b-b04b-ba628938e3bf None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:00.472728 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad152c5f-b2c7-4424-91da-5f2349f27775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:00.473366 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad152c5f-b2c7-4424-91da-5f2349f27775 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:00.473647 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad152c5f-b2c7-4424-91da-5f2349f27775 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:00.473918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad152c5f-b2c7-4424-91da-5f2349f27775 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:00.489006 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae78dd-46b1-46cd-88b2-3a20bf01b274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:00.489650 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae78dd-46b1-46cd-88b2-3a20bf01b274 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:00.489929 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae78dd-46b1-46cd-88b2-3a20bf01b274 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:00.490203 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae78dd-46b1-46cd-88b2-3a20bf01b274 None admin] PATH_INFO: `/v3/projects/255c31b4828b499abeb900572c1ce254` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:00.491510 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae78dd-46b1-46cd-88b2-3a20bf01b274 None admin] RBAC: Authorizing `identity:get_project(project_id=255c31b4828b499abeb900572c1ce254)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:00.492961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae78dd-46b1-46cd-88b2-3a20bf01b274 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:00.494631 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 137/258] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:54:00 2022] GET /identity/v3/projects/255c31b4828b499abeb900572c1ce254 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:00.495412 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae78dd-46b1-46cd-88b2-3a20bf01b274 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:00.499332 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad152c5f-b2c7-4424-91da-5f2349f27775 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:00.509616 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-ad152c5f-b2c7-4424-91da-5f2349f27775 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:54:00.509616 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:00.509616 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:00.509616 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:00.509616 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:00.509616 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:00.509616 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:00.509616 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:00.509616 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:00.509616 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:00.509616 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:00.509616 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:00.509616 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:00.509616 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:00.509616 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:00.509616 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:00.510527 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:00.510527 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:00.510527 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:00.510527 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:00.510527 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:00.510527 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:00.510527 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:00.510527 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:00.510527 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:00.510527 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:00.510527 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:00.510527 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:00.510527 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:00.510527 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:00.510527 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:00.510527 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:00.511363 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:00.511363 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:00.511363 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:00.511363 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:00.511363 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:00.511363 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:00.511363 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:00.511363 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:00.511363 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:00.511363 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:00.511363 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:54:00.511363 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:00.513813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 122/259] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:54:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:00.514822 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad152c5f-b2c7-4424-91da-5f2349f27775 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:00.611250 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:00.634144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad152c5f-b2c7-4424-91da-5f2349f27775 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:00.652084 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:00.699459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e7aa5-b164-4f62-a3ef-4a459c95bdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:00.701162 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e7aa5-b164-4f62-a3ef-4a459c95bdaf None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:00.701851 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e7aa5-b164-4f62-a3ef-4a459c95bdaf None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:00.702625 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e7aa5-b164-4f62-a3ef-4a459c95bdaf None admin] PATH_INFO: `/v3/projects/255c31b4828b499abeb900572c1ce254/users/fbabb1d409a7427f96084ad22c80b8ac/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:00.710682 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae78dd-46b1-46cd-88b2-3a20bf01b274 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:00.713825 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e7aa5-b164-4f62-a3ef-4a459c95bdaf None admin] RBAC: Authorizing `identity:create_grant(project_id=255c31b4828b499abeb900572c1ce254, user_id=fbabb1d409a7427f96084ad22c80b8ac, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:00.716916 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e7aa5-b164-4f62-a3ef-4a459c95bdaf None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:00.761059 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9657d07-03d7-479b-9c70-3614297d27eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:00.762141 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9657d07-03d7-479b-9c70-3614297d27eb None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:00.762678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9657d07-03d7-479b-9c70-3614297d27eb None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:00.763318 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9657d07-03d7-479b-9c70-3614297d27eb None admin] PATH_INFO: `/v3/users/4e13832df24c4448b1ca5a26869875f7` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:00.764803 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9657d07-03d7-479b-9c70-3614297d27eb None admin] RBAC: Authorizing `identity:get_user(user_id=4e13832df24c4448b1ca5a26869875f7)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:00.766450 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9657d07-03d7-479b-9c70-3614297d27eb None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:00.769038 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 138/260] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:54:00 2022] GET /identity/v3/users/4e13832df24c4448b1ca5a26869875f7 => generated 283 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:00.769987 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9657d07-03d7-479b-9c70-3614297d27eb None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:00.806263 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 123/261] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 3 04:54:00 2022] PUT /identity/v3/projects/255c31b4828b499abeb900572c1ce254/users/fbabb1d409a7427f96084ad22c80b8ac/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 04:54:00.807824 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e7aa5-b164-4f62-a3ef-4a459c95bdaf None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:00.937766 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9657d07-03d7-479b-9c70-3614297d27eb None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:00.997354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:01.016158 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f4a9bc-36e9-4033-885a-ed6df72bf82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:01.017879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4a9bc-36e9-4033-885a-ed6df72bf82d None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:01.018396 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4a9bc-36e9-4033-885a-ed6df72bf82d None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:01.018886 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4a9bc-36e9-4033-885a-ed6df72bf82d None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:01.022101 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f4a9bc-36e9-4033-885a-ed6df72bf82d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:01.022686 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f4a9bc-36e9-4033-885a-ed6df72bf82d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:01.024555 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f4a9bc-36e9-4033-885a-ed6df72bf82d None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:01.044490 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 139/262] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:54:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:01.052037 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f4a9bc-36e9-4033-885a-ed6df72bf82d None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:01.101678 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e7aa5-b164-4f62-a3ef-4a459c95bdaf None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:01.162763 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4114e72-6c03-41f0-9252-789c9ea00fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:01.163413 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4114e72-6c03-41f0-9252-789c9ea00fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:01.163696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4114e72-6c03-41f0-9252-789c9ea00fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:01.163978 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4114e72-6c03-41f0-9252-789c9ea00fa6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:01.183482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:01.191052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f4a9bc-36e9-4033-885a-ed6df72bf82d None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:01.229742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4a9254-ae44-4232-bc8f-4474ed2072b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:01.230910 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a9254-ae44-4232-bc8f-4474ed2072b4 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:01.231982 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a9254-ae44-4232-bc8f-4474ed2072b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:01.232461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a9254-ae44-4232-bc8f-4474ed2072b4 None admin] PATH_INFO: `/v3/users/4e13832df24c4448b1ca5a26869875f7` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:01.235024 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4a9254-ae44-4232-bc8f-4474ed2072b4 None admin] RBAC: Authorizing `identity:get_user(user_id=4e13832df24c4448b1ca5a26869875f7)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:01.236813 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4a9254-ae44-4232-bc8f-4474ed2072b4 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:01.239102 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 140/263] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:54:01 2022] GET /identity/v3/users/4e13832df24c4448b1ca5a26869875f7 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:01.245284 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4a9254-ae44-4232-bc8f-4474ed2072b4 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:01.316501 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:01.349198 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4114e72-6c03-41f0-9252-789c9ea00fa6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:01.350685 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4114e72-6c03-41f0-9252-789c9ea00fa6 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:01.362076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4a9254-ae44-4232-bc8f-4474ed2072b4 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:01.406034 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847104fa-6428-48ed-b4b8-274add40f6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:01.407082 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847104fa-6428-48ed-b4b8-274add40f6d0 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:01.407564 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847104fa-6428-48ed-b4b8-274add40f6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:01.408066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847104fa-6428-48ed-b4b8-274add40f6d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:01.483765 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847104fa-6428-48ed-b4b8-274add40f6d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:01.485767 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847104fa-6428-48ed-b4b8-274add40f6d0 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:01.561247 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:01.608324 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 141/264] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:54:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 03 04:54:01.612015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 124/265] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:54:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 804 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:54:01.614504 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847104fa-6428-48ed-b4b8-274add40f6d0 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:01.616960 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4114e72-6c03-41f0-9252-789c9ea00fa6 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:01.705325 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:01.727124 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847104fa-6428-48ed-b4b8-274add40f6d0 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:01.766788 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb629d0-b0d6-4802-a31b-d5691dd61f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:01.767873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb629d0-b0d6-4802-a31b-d5691dd61f96 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:01.768490 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb629d0-b0d6-4802-a31b-d5691dd61f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:01.769009 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb629d0-b0d6-4802-a31b-d5691dd61f96 None admin] PATH_INFO: `/v3/projects/255c31b4828b499abeb900572c1ce254` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:01.770525 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb629d0-b0d6-4802-a31b-d5691dd61f96 None admin] RBAC: Authorizing `identity:get_project(project_id=255c31b4828b499abeb900572c1ce254)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:01.772354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb629d0-b0d6-4802-a31b-d5691dd61f96 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:01.774607 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 142/266] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:54:01 2022] GET /identity/v3/projects/255c31b4828b499abeb900572c1ce254 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:01.780912 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb629d0-b0d6-4802-a31b-d5691dd61f96 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:01.808304 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4114e72-6c03-41f0-9252-789c9ea00fa6 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:01.816479 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:01.894806 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb629d0-b0d6-4802-a31b-d5691dd61f96 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:01.911574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa500115-38ed-441b-ba12-2e3cbdf0059c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:01.912254 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa500115-38ed-441b-ba12-2e3cbdf0059c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:01.912542 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa500115-38ed-441b-ba12-2e3cbdf0059c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:01.912866 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa500115-38ed-441b-ba12-2e3cbdf0059c None admin] PATH_INFO: `/v3/projects/255c31b4828b499abeb900572c1ce254` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:01.914055 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa500115-38ed-441b-ba12-2e3cbdf0059c None admin] RBAC: Authorizing `identity:get_project(project_id=255c31b4828b499abeb900572c1ce254)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:01.915928 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa500115-38ed-441b-ba12-2e3cbdf0059c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:01.917645 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 125/267] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:54:01 2022] GET /identity/v3/projects/255c31b4828b499abeb900572c1ce254 => generated 302 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:01.923364 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff236f4f-8cf5-49ec-b3b7-b1dcc9b7a924 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:01.924193 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff236f4f-8cf5-49ec-b3b7-b1dcc9b7a924 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:01.924474 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff236f4f-8cf5-49ec-b3b7-b1dcc9b7a924 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:01.925890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 126/268] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:01.926520 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff236f4f-8cf5-49ec-b3b7-b1dcc9b7a924 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:01.936257 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d5211-edf1-4aa5-8b8a-780df3a5a94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:01.937193 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d5211-edf1-4aa5-8b8a-780df3a5a94a None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:01.937676 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d5211-edf1-4aa5-8b8a-780df3a5a94a None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:01.938168 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d5211-edf1-4aa5-8b8a-780df3a5a94a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:01.939228 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d5211-edf1-4aa5-8b8a-780df3a5a94a None admin] RBAC: Adding query filter params (role.id=e39d4ecea4c64d5794e1079c9be3ad36, scope.project.id=255c31b4828b499abeb900572c1ce254, user.id=4e13832df24c4448b1ca5a26869875f7) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:01.940278 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d5211-edf1-4aa5-8b8a-780df3a5a94a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:01.942139 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d5211-edf1-4aa5-8b8a-780df3a5a94a None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:01.949393 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 143/269] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:54:01 2022] GET /identity/v3/role_assignments?user.id=4e13832df24c4448b1ca5a26869875f7&scope.project.id=255c31b4828b499abeb900572c1ce254&role.id=e39d4ecea4c64d5794e1079c9be3ad36 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:01.951046 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8497947-8ab2-4230-ba37-8c4cd3be8d5f None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:01.958878 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8497947-8ab2-4230-ba37-8c4cd3be8d5f None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:01.959668 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8497947-8ab2-4230-ba37-8c4cd3be8d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:01.987288 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-d8497947-8ab2-4230-ba37-8c4cd3be8d5f None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:01.988864 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-d8497947-8ab2-4230-ba37-8c4cd3be8d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:02.118128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 144/270] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:02.120180 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85b1af-afa5-40cd-92a2-b9897e7664c3 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:02.121755 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85b1af-afa5-40cd-92a2-b9897e7664c3 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:02.123002 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85b1af-afa5-40cd-92a2-b9897e7664c3 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:02.126355 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 145/271] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:02.128778 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48ea03-924b-46ae-9812-4aead99ae005 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:02.129926 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48ea03-924b-46ae-9812-4aead99ae005 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:02.130599 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48ea03-924b-46ae-9812-4aead99ae005 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:02.133862 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 146/272] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:02.136430 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff236f4f-8cf5-49ec-b3b7-b1dcc9b7a924 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:02.138412 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f471a61-34f2-4c7e-bf9b-712d30dac9c4 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:02.138946 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f471a61-34f2-4c7e-bf9b-712d30dac9c4 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:02.139449 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f471a61-34f2-4c7e-bf9b-712d30dac9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:02.151274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:02.168625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-4f471a61-34f2-4c7e-bf9b-712d30dac9c4 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:02.172166 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-4f471a61-34f2-4c7e-bf9b-712d30dac9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:02.191516 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408859e-3805-4751-a484-4b90647acbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:02.192703 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408859e-3805-4751-a484-4b90647acbd1 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:02.193868 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408859e-3805-4751-a484-4b90647acbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:02.194151 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408859e-3805-4751-a484-4b90647acbd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:02.195011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4408859e-3805-4751-a484-4b90647acbd1 None admin] RBAC: Adding query filter params (role.id=561445e71aff40a0af7b01c539d3cfd9, scope.project.id=255c31b4828b499abeb900572c1ce254, user.id=4e13832df24c4448b1ca5a26869875f7) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:02.195314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4408859e-3805-4751-a484-4b90647acbd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:02.196782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4408859e-3805-4751-a484-4b90647acbd1 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:02.198316 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:02.203937 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 127/273] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:54:01 2022] GET /identity/v3/role_assignments?user.id=4e13832df24c4448b1ca5a26869875f7&scope.project.id=255c31b4828b499abeb900572c1ce254&role.id=561445e71aff40a0af7b01c539d3cfd9 => generated 608 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:02.205155 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eae675-e61d-4a85-8f25-dcf5832ae1f4 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:02.205560 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eae675-e61d-4a85-8f25-dcf5832ae1f4 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:02.205816 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eae675-e61d-4a85-8f25-dcf5832ae1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:02.242233 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-b8eae675-e61d-4a85-8f25-dcf5832ae1f4 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:02.244347 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b8eae675-e61d-4a85-8f25-dcf5832ae1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:02.265221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 147/274] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:02.266136 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f471a61-34f2-4c7e-bf9b-712d30dac9c4 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:02.338073 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 128/275] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:02.339618 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eae675-e61d-4a85-8f25-dcf5832ae1f4 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:02.398076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f471a61-34f2-4c7e-bf9b-712d30dac9c4 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:02.442240 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bcb190-4ea7-402d-861e-0d9c0b971425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:02.442815 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bcb190-4ea7-402d-861e-0d9c0b971425 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:02.443125 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bcb190-4ea7-402d-861e-0d9c0b971425 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:02.443400 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bcb190-4ea7-402d-861e-0d9c0b971425 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:02.444559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bcb190-4ea7-402d-861e-0d9c0b971425 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:02.446022 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bcb190-4ea7-402d-861e-0d9c0b971425 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:02.448492 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 148/276] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:02.448492 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bcb190-4ea7-402d-861e-0d9c0b971425 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:02.478527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:02.512745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eae675-e61d-4a85-8f25-dcf5832ae1f4 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:02.514634 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:02.552686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc1bcc-bfff-4148-b175-ed368e813634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:02.553330 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc1bcc-bfff-4148-b175-ed368e813634 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:02.553608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc1bcc-bfff-4148-b175-ed368e813634 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:02.553881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc1bcc-bfff-4148-b175-ed368e813634 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:02.560024 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bcb190-4ea7-402d-861e-0d9c0b971425 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:02.567994 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc1bcc-bfff-4148-b175-ed368e813634 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:02.572866 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-90bc1bcc-bfff-4148-b175-ed368e813634 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 04:54:02.572866 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:02.572866 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:02.572866 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:02.572866 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:02.572866 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:02.572866 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:02.572866 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:02.572866 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:02.572866 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:02.572866 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:02.572866 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:02.572866 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:02.572866 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:02.572866 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:02.572866 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:02.573747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:02.573747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:02.573747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:02.573747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:02.573747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:02.573747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:02.573747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:02.573747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:02.573747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:02.573747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:02.573747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:02.573747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:02.573747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:02.573747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:02.573747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:02.573747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:02.574541 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:02.574541 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:02.574541 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:02.574541 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:02.574541 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:02.574541 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:02.574541 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:02.574541 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:02.574541 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:02.574541 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:02.574541 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 04:54:02.574541 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:02.574541 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 129/277] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:54:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:02.575782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc1bcc-bfff-4148-b175-ed368e813634 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:02.606907 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb1174-249f-42bd-929a-14d899fcbfe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:02.607547 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb1174-249f-42bd-929a-14d899fcbfe8 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:02.607833 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb1174-249f-42bd-929a-14d899fcbfe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:02.608107 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb1174-249f-42bd-929a-14d899fcbfe8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:02.619035 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cb1174-249f-42bd-929a-14d899fcbfe8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:02.623893 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-83cb1174-249f-42bd-929a-14d899fcbfe8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:54:02.623893 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:02.623893 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:02.623893 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:02.623893 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:02.623893 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:02.623893 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:02.623893 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:02.623893 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:02.623893 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:02.623893 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:02.623893 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:02.623893 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:02.623893 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:02.623893 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:02.623893 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:02.624822 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:02.624822 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:02.624822 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:02.624822 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:02.624822 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:02.624822 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:02.624822 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:02.624822 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:02.624822 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:02.624822 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:02.624822 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:02.624822 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:02.624822 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:02.624822 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:02.624822 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:02.624822 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:02.625759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:02.625759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:02.625759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:02.625759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:02.625759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:02.625759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:02.625759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:02.625759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:02.625759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:02.625759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:02.625759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:54:02.625759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:02.625759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 149/278] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:54:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:02.627159 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb1174-249f-42bd-929a-14d899fcbfe8 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:02.695445 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc1bcc-bfff-4148-b175-ed368e813634 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:02.723557 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:02.734991 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea1b12b-a0da-4a3d-8433-0bfd98098ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:02.737676 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1b12b-a0da-4a3d-8433-0bfd98098ee8 None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:02.737676 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1b12b-a0da-4a3d-8433-0bfd98098ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:02.737676 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1b12b-a0da-4a3d-8433-0bfd98098ee8 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:02.737676 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea1b12b-a0da-4a3d-8433-0bfd98098ee8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:02.739688 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea1b12b-a0da-4a3d-8433-0bfd98098ee8 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:02.742127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb1174-249f-42bd-929a-14d899fcbfe8 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:02.779399 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45767a5-aa10-40a1-b735-62d02912c5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:02.780030 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45767a5-aa10-40a1-b735-62d02912c5fc None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:02.780366 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45767a5-aa10-40a1-b735-62d02912c5fc None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:02.780640 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45767a5-aa10-40a1-b735-62d02912c5fc None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:02.781345 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45767a5-aa10-40a1-b735-62d02912c5fc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:02.781656 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45767a5-aa10-40a1-b735-62d02912c5fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:02.783620 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45767a5-aa10-40a1-b735-62d02912c5fc None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:02.791794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 150/279] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 3 04:54:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:02.792569 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45767a5-aa10-40a1-b735-62d02912c5fc None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:02.844948 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 130/280] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 3 04:54:02 2022] POST /identity/v3/users => generated 293 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:54:02.844948 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea1b12b-a0da-4a3d-8433-0bfd98098ee8 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:02.878264 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:02.925732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45767a5-aa10-40a1-b735-62d02912c5fc None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:02.973292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd64a93-17e8-4643-8575-b441dbce6a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:02.974249 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd64a93-17e8-4643-8575-b441dbce6a2f None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:02.974717 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd64a93-17e8-4643-8575-b441dbce6a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:02.975206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd64a93-17e8-4643-8575-b441dbce6a2f None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:02.976234 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd64a93-17e8-4643-8575-b441dbce6a2f None admin] RBAC: Adding query filter params (name=member) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:02.976738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd64a93-17e8-4643-8575-b441dbce6a2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:02.978566 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd64a93-17e8-4643-8575-b441dbce6a2f None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:02.989304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 151/281] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 3 04:54:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:02.997421 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea1b12b-a0da-4a3d-8433-0bfd98098ee8 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:02.997616 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd64a93-17e8-4643-8575-b441dbce6a2f None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:03.005110 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:03.045602 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4601fd-6433-4a79-8cb8-63b9c3fa9b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:03.046271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4601fd-6433-4a79-8cb8-63b9c3fa9b34 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:03.046553 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4601fd-6433-4a79-8cb8-63b9c3fa9b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:03.046832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4601fd-6433-4a79-8cb8-63b9c3fa9b34 None admin] PATH_INFO: `/v3/users/e9de1de3860a40fbabbea9687378bcda` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:03.048059 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4601fd-6433-4a79-8cb8-63b9c3fa9b34 None admin] RBAC: Authorizing `identity:get_user(user_id=e9de1de3860a40fbabbea9687378bcda)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:03.049558 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4601fd-6433-4a79-8cb8-63b9c3fa9b34 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:03.056597 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 131/282] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:54:02 2022] GET /identity/v3/users/e9de1de3860a40fbabbea9687378bcda => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:03.057966 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4601fd-6433-4a79-8cb8-63b9c3fa9b34 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:03.124876 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd64a93-17e8-4643-8575-b441dbce6a2f None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:03.173405 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11941e8b-a7fa-4494-9087-90ffb94a145f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:03.175015 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11941e8b-a7fa-4494-9087-90ffb94a145f None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:03.175503 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11941e8b-a7fa-4494-9087-90ffb94a145f None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:03.175975 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11941e8b-a7fa-4494-9087-90ffb94a145f None admin] PATH_INFO: `/v3/users/95bff512cba344fd9d61ab4a7d324814` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:03.179632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:03.195054 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11941e8b-a7fa-4494-9087-90ffb94a145f None admin] RBAC: Authorizing `identity:get_user(user_id=95bff512cba344fd9d61ab4a7d324814)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:03.196773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11941e8b-a7fa-4494-9087-90ffb94a145f None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:03.198256 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4601fd-6433-4a79-8cb8-63b9c3fa9b34 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:03.198794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 152/283] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:54:02 2022] GET /identity/v3/users/95bff512cba344fd9d61ab4a7d324814 => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:03.205413 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11941e8b-a7fa-4494-9087-90ffb94a145f None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:03.244028 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71073dd5-4dd0-46ca-81df-0c172b8b127a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:03.244682 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71073dd5-4dd0-46ca-81df-0c172b8b127a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:03.244968 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71073dd5-4dd0-46ca-81df-0c172b8b127a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:03.245245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71073dd5-4dd0-46ca-81df-0c172b8b127a None admin] PATH_INFO: `/v3/projects/255c31b4828b499abeb900572c1ce254` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:03.246482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71073dd5-4dd0-46ca-81df-0c172b8b127a None admin] RBAC: Authorizing `identity:get_project(project_id=255c31b4828b499abeb900572c1ce254)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:03.248034 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71073dd5-4dd0-46ca-81df-0c172b8b127a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:03.249955 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 132/284] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:54:03 2022] GET /identity/v3/projects/255c31b4828b499abeb900572c1ce254 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:03.256035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71073dd5-4dd0-46ca-81df-0c172b8b127a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:03.296012 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:03.370344 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11941e8b-a7fa-4494-9087-90ffb94a145f None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:03.427892 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71073dd5-4dd0-46ca-81df-0c172b8b127a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:03.449204 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26494c-195b-41c8-ad6d-1885ba91cc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:03.449839 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26494c-195b-41c8-ad6d-1885ba91cc2b None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:03.450122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26494c-195b-41c8-ad6d-1885ba91cc2b None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:03.450398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26494c-195b-41c8-ad6d-1885ba91cc2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:03.451224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26494c-195b-41c8-ad6d-1885ba91cc2b None admin] RBAC: Adding query filter params (role.id=b0e0084bb31348a582616b1421f3b10c, user.id=95bff512cba344fd9d61ab4a7d324814, scope.system=all) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:03.451539 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26494c-195b-41c8-ad6d-1885ba91cc2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:03.453004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26494c-195b-41c8-ad6d-1885ba91cc2b None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:03.466956 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 153/285] 10.222.0.40 () {56 vars in 1308 bytes} [Wed Aug 3 04:54:03 2022] GET /identity/v3/role_assignments?user.id=95bff512cba344fd9d61ab4a7d324814&scope.system=all&role.id=b0e0084bb31348a582616b1421f3b10c => generated 226 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:03.508153 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de9812-1d1e-4498-95f6-133f87e363fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:03.508357 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de9812-1d1e-4498-95f6-133f87e363fe None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:03.508639 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de9812-1d1e-4498-95f6-133f87e363fe None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:03.508919 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de9812-1d1e-4498-95f6-133f87e363fe None admin] PATH_INFO: `/v3/projects/255c31b4828b499abeb900572c1ce254/users/e9de1de3860a40fbabbea9687378bcda/roles/d7a7654a12dd4bd38542203bde537922` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:03.515477 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de9812-1d1e-4498-95f6-133f87e363fe None admin] RBAC: Authorizing `identity:create_grant(project_id=255c31b4828b499abeb900572c1ce254, user_id=e9de1de3860a40fbabbea9687378bcda, role_id=d7a7654a12dd4bd38542203bde537922)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:03.515477 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de9812-1d1e-4498-95f6-133f87e363fe None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:03.549749 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ed4d5-f5f4-4288-acbb-790e49555df8 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:03.550145 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ed4d5-f5f4-4288-acbb-790e49555df8 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:03.550501 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ed4d5-f5f4-4288-acbb-790e49555df8 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:03.552293 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 154/286] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:03.558124 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db551f5d-0c29-42c7-bb3e-e499ceb590e8 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:03.558715 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db551f5d-0c29-42c7-bb3e-e499ceb590e8 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:03.559010 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db551f5d-0c29-42c7-bb3e-e499ceb590e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:03.579939 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-db551f5d-0c29-42c7-bb3e-e499ceb590e8 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:03.581475 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-db551f5d-0c29-42c7-bb3e-e499ceb590e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:03.598764 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 133/287] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 3 04:54:03 2022] PUT /identity/v3/projects/255c31b4828b499abeb900572c1ce254/users/e9de1de3860a40fbabbea9687378bcda/roles/d7a7654a12dd4bd38542203bde537922 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 04:54:03.692121 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 155/288] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 04:54:03.726983 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de9812-1d1e-4498-95f6-133f87e363fe None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:03.750155 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:03.848012 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de9812-1d1e-4498-95f6-133f87e363fe None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:03.884173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2cfc58-aabb-43f3-824c-2bf427b2354a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:03.884900 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2cfc58-aabb-43f3-824c-2bf427b2354a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:03.885145 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2cfc58-aabb-43f3-824c-2bf427b2354a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:03.885420 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2cfc58-aabb-43f3-824c-2bf427b2354a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:03.896106 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2cfc58-aabb-43f3-824c-2bf427b2354a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:03.901377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-7f2cfc58-aabb-43f3-824c-2bf427b2354a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:54:03.901377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:03.901377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:03.901377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:03.901377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:03.901377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:03.901377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:03.901377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:03.901377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:03.901377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:03.901377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:03.901377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:03.901377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:03.901377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:03.901377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:03.901377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:03.902255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:03.902255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:03.902255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:03.902255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:03.902255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:03.902255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:03.902255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:03.902255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:03.902255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:03.902255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:03.902255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:03.902255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:03.902255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:03.902255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:03.902255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:03.902255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:03.903066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:03.903066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:03.903066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:03.903066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:03.903066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:03.903066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:03.903066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:03.903066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:03.903066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:03.903066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:03.903066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:54:03.903066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:03.903066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 134/289] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:54:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:03.908974 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db551f5d-0c29-42c7-bb3e-e499ceb590e8 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:04.004747 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:04.067170 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db551f5d-0c29-42c7-bb3e-e499ceb590e8 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:04.117476 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eadef14-9ffb-4757-b852-ed52494b3c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:04.118030 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eadef14-9ffb-4757-b852-ed52494b3c01 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:04.118315 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eadef14-9ffb-4757-b852-ed52494b3c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:04.118594 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eadef14-9ffb-4757-b852-ed52494b3c01 None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:04.119353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eadef14-9ffb-4757-b852-ed52494b3c01 None admin] RBAC: Adding query filter params (name=member) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:04.119661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eadef14-9ffb-4757-b852-ed52494b3c01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:04.121047 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eadef14-9ffb-4757-b852-ed52494b3c01 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:04.134336 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 156/290] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 3 04:54:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:04.139607 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2cfc58-aabb-43f3-824c-2bf427b2354a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:04.268485 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:04.307648 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2cfc58-aabb-43f3-824c-2bf427b2354a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:04.355463 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e871f-9621-4a97-872f-834bd5ce64de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:04.356550 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e871f-9621-4a97-872f-834bd5ce64de None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:04.357063 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e871f-9621-4a97-872f-834bd5ce64de None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:04.357542 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e871f-9621-4a97-872f-834bd5ce64de None admin] PATH_INFO: `/v3/users/fbabb1d409a7427f96084ad22c80b8ac` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:04.359099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e871f-9621-4a97-872f-834bd5ce64de None admin] RBAC: Authorizing `identity:get_user(user_id=fbabb1d409a7427f96084ad22c80b8ac)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:04.360782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e871f-9621-4a97-872f-834bd5ce64de None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:04.362967 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 135/291] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:54:04 2022] GET /identity/v3/users/fbabb1d409a7427f96084ad22c80b8ac => generated 297 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:04.372366 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eadef14-9ffb-4757-b852-ed52494b3c01 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:04.494873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eadef14-9ffb-4757-b852-ed52494b3c01 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:04.523223 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:04.542520 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31eb4f-1650-47a9-a30c-0e8a334c9c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:04.543198 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31eb4f-1650-47a9-a30c-0e8a334c9c6a None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:04.543480 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31eb4f-1650-47a9-a30c-0e8a334c9c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:04.543752 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31eb4f-1650-47a9-a30c-0e8a334c9c6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:04.634438 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31eb4f-1650-47a9-a30c-0e8a334c9c6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:04.635941 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31eb4f-1650-47a9-a30c-0e8a334c9c6a None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:04.789969 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 157/292] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:54:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:54:04.795882 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e871f-9621-4a97-872f-834bd5ce64de None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:04.919135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e871f-9621-4a97-872f-834bd5ce64de None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:04.969200 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e50bc-efab-40d5-9d8c-b29a66b37af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:04.969847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e50bc-efab-40d5-9d8c-b29a66b37af1 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:04.970128 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e50bc-efab-40d5-9d8c-b29a66b37af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:04.970430 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e50bc-efab-40d5-9d8c-b29a66b37af1 None admin] PATH_INFO: `/v3/projects/255c31b4828b499abeb900572c1ce254` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:04.974692 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e50bc-efab-40d5-9d8c-b29a66b37af1 None admin] RBAC: Authorizing `identity:get_project(project_id=255c31b4828b499abeb900572c1ce254)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:04.976177 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e50bc-efab-40d5-9d8c-b29a66b37af1 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:04.978206 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 136/293] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:54:04 2022] GET /identity/v3/projects/255c31b4828b499abeb900572c1ce254 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:04.984120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31eb4f-1650-47a9-a30c-0e8a334c9c6a None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:05.010429 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:05.118542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31eb4f-1650-47a9-a30c-0e8a334c9c6a None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:05.153222 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d020f-35a7-4fb2-98cd-4792b4c0ec4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:05.153848 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d020f-35a7-4fb2-98cd-4792b4c0ec4a None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:05.154128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d020f-35a7-4fb2-98cd-4792b4c0ec4a None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:05.154401 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d020f-35a7-4fb2-98cd-4792b4c0ec4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:05.155218 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d020f-35a7-4fb2-98cd-4792b4c0ec4a None admin] RBAC: Adding query filter params (role.id=b0e0084bb31348a582616b1421f3b10c, scope.project.id=255c31b4828b499abeb900572c1ce254, user.id=fbabb1d409a7427f96084ad22c80b8ac) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:05.155530 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d020f-35a7-4fb2-98cd-4792b4c0ec4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:05.156975 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d020f-35a7-4fb2-98cd-4792b4c0ec4a None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:05.164388 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 158/294] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:54:04 2022] GET /identity/v3/role_assignments?user.id=fbabb1d409a7427f96084ad22c80b8ac&scope.project.id=255c31b4828b499abeb900572c1ce254&role.id=b0e0084bb31348a582616b1421f3b10c => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:05.584376 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831eea8d-7a35-438d-b15e-43cbb6811fe9 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:05.584853 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831eea8d-7a35-438d-b15e-43cbb6811fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:05.584958 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831eea8d-7a35-438d-b15e-43cbb6811fe9 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:05.586282 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 137/295] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:05.591515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b05b90-0aa7-4c8b-b951-e83c3089140c None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:05.591807 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b05b90-0aa7-4c8b-b951-e83c3089140c None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:05.592093 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b05b90-0aa7-4c8b-b951-e83c3089140c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:05.597611 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ecea1-5620-40e0-bb73-6785b69f6ea4 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:05.597896 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ecea1-5620-40e0-bb73-6785b69f6ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:05.598179 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ecea1-5620-40e0-bb73-6785b69f6ea4 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:05.599832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 138/296] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:54:05.604744 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ef30f-fd75-4371-9ffc-2d8386c84507 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:05.605027 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ef30f-fd75-4371-9ffc-2d8386c84507 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:05.605307 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ef30f-fd75-4371-9ffc-2d8386c84507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:05.612750 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-b9b05b90-0aa7-4c8b-b951-e83c3089140c None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:05.614806 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-b9b05b90-0aa7-4c8b-b951-e83c3089140c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:05.627514 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-f59ef30f-fd75-4371-9ffc-2d8386c84507 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:05.628735 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f59ef30f-fd75-4371-9ffc-2d8386c84507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:05.630824 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:05.714157 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 159/297] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:05.726517 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 139/298] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 04:54:05.740516 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b05b90-0aa7-4c8b-b951-e83c3089140c None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:05.754519 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ef30f-fd75-4371-9ffc-2d8386c84507 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:05.762649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:05.865102 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b05b90-0aa7-4c8b-b951-e83c3089140c None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:05.868892 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ef30f-fd75-4371-9ffc-2d8386c84507 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:05.903963 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a9afc-5ccf-447a-8b8f-e97a48647975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:05.905051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a9afc-5ccf-447a-8b8f-e97a48647975 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:05.905510 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a9afc-5ccf-447a-8b8f-e97a48647975 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:05.905947 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a9afc-5ccf-447a-8b8f-e97a48647975 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:05.906059 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323bfb87-9eaf-4852-96c7-b814d2f5d64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:05.906606 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bfb87-9eaf-4852-96c7-b814d2f5d64a None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:05.906915 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bfb87-9eaf-4852-96c7-b814d2f5d64a None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:05.907194 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bfb87-9eaf-4852-96c7-b814d2f5d64a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:05.908970 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323bfb87-9eaf-4852-96c7-b814d2f5d64a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:05.910419 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323bfb87-9eaf-4852-96c7-b814d2f5d64a None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:05.912056 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 160/299] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:05.913324 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb843252-b645-4813-a65c-dbb68cf1b675 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:05.913611 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb843252-b645-4813-a65c-dbb68cf1b675 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:05.913883 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb843252-b645-4813-a65c-dbb68cf1b675 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:05.915367 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 161/300] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:05.918869 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:05.920819 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb843252-b645-4813-a65c-dbb68cf1b675 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:05.923236 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662a9afc-5ccf-447a-8b8f-e97a48647975 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:05.928783 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-662a9afc-5ccf-447a-8b8f-e97a48647975 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 04:54:05.928783 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:05.928783 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:05.928783 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:05.928783 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:05.928783 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:05.928783 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:05.928783 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:05.928783 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:05.928783 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:05.928783 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:05.928783 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:05.928783 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:05.928783 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:05.928783 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:05.928783 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:05.934142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:05.934142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:05.934142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:05.934142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:05.934142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:05.934142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:05.934142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:05.934142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:05.934142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:05.934142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:05.934142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:05.934142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:05.934142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:05.934142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:05.934142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:05.934142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:05.935624 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:05.935624 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:05.935624 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:05.935624 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:05.935624 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:05.935624 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:05.935624 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:05.935624 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:05.935624 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:05.935624 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:05.935624 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 04:54:05.935624 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:05.935624 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 140/301] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:54:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:54:05.935624 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a5f3e-5b9f-4dfb-802c-15444062b072 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:05.936322 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a5f3e-5b9f-4dfb-802c-15444062b072 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:05.936322 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a5f3e-5b9f-4dfb-802c-15444062b072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:05.951338 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-812a5f3e-5b9f-4dfb-802c-15444062b072 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:05.953147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-812a5f3e-5b9f-4dfb-802c-15444062b072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:06.014430 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:06.024639 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 141/302] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:06.025620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a5f3e-5b9f-4dfb-802c-15444062b072 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:06.039144 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb843252-b645-4813-a65c-dbb68cf1b675 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:06.082240 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f5dfa-0368-412a-a0ab-5cd7209f4e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:06.082901 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f5dfa-0368-412a-a0ab-5cd7209f4e72 None admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:06.083189 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f5dfa-0368-412a-a0ab-5cd7209f4e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:06.083439 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f5dfa-0368-412a-a0ab-5cd7209f4e72 None admin] PATH_INFO: `/v3/groups` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:06.084103 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f5dfa-0368-412a-a0ab-5cd7209f4e72 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:06.085657 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f5dfa-0368-412a-a0ab-5cd7209f4e72 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:06.141243 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a5f3e-5b9f-4dfb-802c-15444062b072 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:06.159711 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 162/303] 10.222.0.40 () {60 vars in 1134 bytes} [Wed Aug 3 04:54:05 2022] POST /identity/v3/groups => generated 229 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:54:06.160459 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f5dfa-0368-412a-a0ab-5cd7209f4e72 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:06.181670 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:06.195181 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566fcc6-74d1-486b-80b2-3d39eec76cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:06.196876 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566fcc6-74d1-486b-80b2-3d39eec76cbd None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:06.197569 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566fcc6-74d1-486b-80b2-3d39eec76cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:06.198229 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566fcc6-74d1-486b-80b2-3d39eec76cbd None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:06.199723 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9566fcc6-74d1-486b-80b2-3d39eec76cbd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:06.200569 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9566fcc6-74d1-486b-80b2-3d39eec76cbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:06.203167 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9566fcc6-74d1-486b-80b2-3d39eec76cbd None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:06.220360 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 142/304] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 3 04:54:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:06.231688 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566fcc6-74d1-486b-80b2-3d39eec76cbd None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:06.306809 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f5dfa-0368-412a-a0ab-5cd7209f4e72 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:06.345928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9446809e-788b-457d-800e-8ed17ab74742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:06.346887 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446809e-788b-457d-800e-8ed17ab74742 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:06.346887 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446809e-788b-457d-800e-8ed17ab74742 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:06.347150 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446809e-788b-457d-800e-8ed17ab74742 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:06.353399 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566fcc6-74d1-486b-80b2-3d39eec76cbd None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:06.359674 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9446809e-788b-457d-800e-8ed17ab74742 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:06.365238 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-9446809e-788b-457d-800e-8ed17ab74742 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:54:06.365238 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:06.365238 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:06.365238 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:06.365238 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:06.365238 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:06.365238 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:06.365238 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:06.365238 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:06.365238 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:06.365238 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:06.365238 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:06.365238 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:06.365238 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:06.365238 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:06.365238 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:06.366324 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:06.366324 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:06.366324 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:06.366324 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:06.366324 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:06.366324 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:06.366324 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:06.366324 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:06.366324 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:06.366324 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:06.366324 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:06.366324 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:06.366324 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:06.366324 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:06.366324 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:06.366324 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:06.367409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:06.367409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:06.367409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:06.367409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:06.367409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:06.367409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:06.367409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:06.367409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:06.367409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:06.367409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:06.367409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:54:06.367409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:06.367409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 163/305] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:54:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:06.368421 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e5f11-b296-4373-ab2b-762ecf002969 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:06.368704 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e5f11-b296-4373-ab2b-762ecf002969 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:06.368979 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e5f11-b296-4373-ab2b-762ecf002969 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:06.370363 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 164/306] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:06.372894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e5f11-b296-4373-ab2b-762ecf002969 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:06.376920 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:06.412327 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae93c8-2ada-49f7-af91-ea4edfad2863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:06.412930 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae93c8-2ada-49f7-af91-ea4edfad2863 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:06.413210 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae93c8-2ada-49f7-af91-ea4edfad2863 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:06.413482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae93c8-2ada-49f7-af91-ea4edfad2863 None admin] PATH_INFO: `/v3/users/99ee0cd592f34497809101d2675b2772` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:06.436113 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae93c8-2ada-49f7-af91-ea4edfad2863 None admin] RBAC: Authorizing `identity:get_user(user_id=99ee0cd592f34497809101d2675b2772)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:06.437897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae93c8-2ada-49f7-af91-ea4edfad2863 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:06.444504 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 143/307] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:54:06 2022] GET /identity/v3/users/99ee0cd592f34497809101d2675b2772 => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:06.444504 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b47237-6944-454c-86b1-fe9fcc58cf6f None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:06.444504 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b47237-6944-454c-86b1-fe9fcc58cf6f None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:06.444504 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b47237-6944-454c-86b1-fe9fcc58cf6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:06.467268 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-d3b47237-6944-454c-86b1-fe9fcc58cf6f None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:06.469208 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d3b47237-6944-454c-86b1-fe9fcc58cf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:06.499959 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e5f11-b296-4373-ab2b-762ecf002969 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:06.536138 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23496aa0-69d1-450d-a228-ac85aac3ec2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:06.536768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23496aa0-69d1-450d-a228-ac85aac3ec2d None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:06.537056 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23496aa0-69d1-450d-a228-ac85aac3ec2d None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:06.537334 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23496aa0-69d1-450d-a228-ac85aac3ec2d None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:06.538053 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23496aa0-69d1-450d-a228-ac85aac3ec2d None admin] RBAC: Adding query filter params (name=member) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:06.538484 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23496aa0-69d1-450d-a228-ac85aac3ec2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:06.539927 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23496aa0-69d1-450d-a228-ac85aac3ec2d None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:06.544314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:06.553376 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 165/308] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 3 04:54:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:06.553376 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23496aa0-69d1-450d-a228-ac85aac3ec2d None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:06.580453 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 144/309] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:06.581685 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b47237-6944-454c-86b1-fe9fcc58cf6f None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:06.638175 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:06.682780 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23496aa0-69d1-450d-a228-ac85aac3ec2d None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:06.710101 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b47237-6944-454c-86b1-fe9fcc58cf6f None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:06.720961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab91c07-3ab6-479a-8e85-94ac7a9098ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:06.721665 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab91c07-3ab6-479a-8e85-94ac7a9098ae None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:06.721955 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab91c07-3ab6-479a-8e85-94ac7a9098ae None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:06.722234 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab91c07-3ab6-479a-8e85-94ac7a9098ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:06.723047 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab91c07-3ab6-479a-8e85-94ac7a9098ae None admin] RBAC: Adding query filter params (role.id=d7a7654a12dd4bd38542203bde537922, user.id=99ee0cd592f34497809101d2675b2772, scope.system=all) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:06.723363 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab91c07-3ab6-479a-8e85-94ac7a9098ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:06.724809 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab91c07-3ab6-479a-8e85-94ac7a9098ae None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:06.730187 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 166/310] 10.222.0.40 () {56 vars in 1308 bytes} [Wed Aug 3 04:54:06 2022] GET /identity/v3/role_assignments?user.id=99ee0cd592f34497809101d2675b2772&scope.system=all&role.id=d7a7654a12dd4bd38542203bde537922 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:06.739930 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab91c07-3ab6-479a-8e85-94ac7a9098ae None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:06.755151 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605097e0-e112-42bd-a772-1aee96cd6481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:06.756199 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605097e0-e112-42bd-a772-1aee96cd6481 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:06.756638 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605097e0-e112-42bd-a772-1aee96cd6481 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:06.757342 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605097e0-e112-42bd-a772-1aee96cd6481 None admin] PATH_INFO: `/v3/users/95bff512cba344fd9d61ab4a7d324814` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:06.759775 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605097e0-e112-42bd-a772-1aee96cd6481 None admin] RBAC: Authorizing `identity:get_user(user_id=95bff512cba344fd9d61ab4a7d324814)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:06.761745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605097e0-e112-42bd-a772-1aee96cd6481 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:06.764612 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 145/311] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:54:06 2022] GET /identity/v3/users/95bff512cba344fd9d61ab4a7d324814 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:06.770642 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605097e0-e112-42bd-a772-1aee96cd6481 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:06.840679 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:06.867826 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab91c07-3ab6-479a-8e85-94ac7a9098ae None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:06.900586 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:06.915018 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605097e0-e112-42bd-a772-1aee96cd6481 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:06.915599 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827657d9-d2ba-47fc-b75d-c4245ed59226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:06.916689 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827657d9-d2ba-47fc-b75d-c4245ed59226 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:06.917139 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827657d9-d2ba-47fc-b75d-c4245ed59226 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:06.918510 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827657d9-d2ba-47fc-b75d-c4245ed59226 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:06.932946 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827657d9-d2ba-47fc-b75d-c4245ed59226 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:06.939856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-827657d9-d2ba-47fc-b75d-c4245ed59226 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 04:54:06.939856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:06.939856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:06.939856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:06.939856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:06.939856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:06.939856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:06.939856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:06.939856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:06.939856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:06.939856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:06.939856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:06.939856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:06.939856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:06.939856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:06.939856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:06.940970 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:06.940970 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:06.940970 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:06.940970 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:06.940970 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:06.940970 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:06.940970 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:06.940970 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:06.940970 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:06.940970 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:06.940970 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:06.940970 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:06.940970 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:06.940970 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:06.940970 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:06.940970 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:06.941989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:06.941989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:06.941989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:06.941989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:06.941989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:06.941989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:06.941989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:06.941989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:06.941989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:06.941989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:06.941989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 04:54:06.941989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:06.949147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 167/312] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:54:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:06.951215 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827657d9-d2ba-47fc-b75d-c4245ed59226 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:06.962806 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e7518-5cda-4a0a-8434-c75cefa8a0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:06.964215 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e7518-5cda-4a0a-8434-c75cefa8a0b3 None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:06.964727 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e7518-5cda-4a0a-8434-c75cefa8a0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:06.965235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e7518-5cda-4a0a-8434-c75cefa8a0b3 None admin] PATH_INFO: `/v3/system/users/95bff512cba344fd9d61ab4a7d324814/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:06.967047 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e7518-5cda-4a0a-8434-c75cefa8a0b3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=95bff512cba344fd9d61ab4a7d324814, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:06.968651 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e7518-5cda-4a0a-8434-c75cefa8a0b3 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:07.043588 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 146/313] 10.222.0.40 () {58 vars in 1334 bytes} [Wed Aug 3 04:54:06 2022] PUT /identity/v3/system/users/95bff512cba344fd9d61ab4a7d324814/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 04:54:07.072746 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827657d9-d2ba-47fc-b75d-c4245ed59226 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:07.114187 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a844f5-d036-401e-93e2-91ac2af72b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:07.114901 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a844f5-d036-401e-93e2-91ac2af72b6a None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:07.115193 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a844f5-d036-401e-93e2-91ac2af72b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:07.115474 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a844f5-d036-401e-93e2-91ac2af72b6a None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:07.116261 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a844f5-d036-401e-93e2-91ac2af72b6a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:07.116576 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a844f5-d036-401e-93e2-91ac2af72b6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:07.118056 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a844f5-d036-401e-93e2-91ac2af72b6a None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:07.128584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 168/314] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 3 04:54:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:07.134294 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e7518-5cda-4a0a-8434-c75cefa8a0b3 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:07.252761 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e7518-5cda-4a0a-8434-c75cefa8a0b3 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:07.264724 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:07.290682 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ebdd4c-e19f-4d45-b6ca-336665655045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:07.291382 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ebdd4c-e19f-4d45-b6ca-336665655045 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:07.291669 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ebdd4c-e19f-4d45-b6ca-336665655045 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:07.291954 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ebdd4c-e19f-4d45-b6ca-336665655045 None admin] PATH_INFO: `/v3/users/e9de1de3860a40fbabbea9687378bcda` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:07.293210 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ebdd4c-e19f-4d45-b6ca-336665655045 None admin] RBAC: Authorizing `identity:get_user(user_id=e9de1de3860a40fbabbea9687378bcda)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:07.294790 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ebdd4c-e19f-4d45-b6ca-336665655045 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:07.296747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 147/315] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:54:07 2022] GET /identity/v3/users/e9de1de3860a40fbabbea9687378bcda => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:07.302284 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a844f5-d036-401e-93e2-91ac2af72b6a None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:07.358612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:07.412744 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a844f5-d036-401e-93e2-91ac2af72b6a None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:07.460949 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de55496-f137-4460-8a88-9219e50c1336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:07.461595 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de55496-f137-4460-8a88-9219e50c1336 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:07.461883 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de55496-f137-4460-8a88-9219e50c1336 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:07.462162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de55496-f137-4460-8a88-9219e50c1336 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:07.536867 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de55496-f137-4460-8a88-9219e50c1336 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:07.538254 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de55496-f137-4460-8a88-9219e50c1336 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:07.598880 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:07.656371 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 169/316] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:54:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:54:07.661972 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ebdd4c-e19f-4d45-b6ca-336665655045 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:07.804051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ebdd4c-e19f-4d45-b6ca-336665655045 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:07.842348 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d66fd-3b1f-41f3-aff3-4959e5213882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:07.843040 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d66fd-3b1f-41f3-aff3-4959e5213882 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:07.843331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d66fd-3b1f-41f3-aff3-4959e5213882 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:07.843614 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d66fd-3b1f-41f3-aff3-4959e5213882 None admin] PATH_INFO: `/v3/projects/255c31b4828b499abeb900572c1ce254` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:07.844797 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d66fd-3b1f-41f3-aff3-4959e5213882 None admin] RBAC: Authorizing `identity:get_project(project_id=255c31b4828b499abeb900572c1ce254)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:07.846351 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d66fd-3b1f-41f3-aff3-4959e5213882 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:07.848368 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 148/317] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:54:07 2022] GET /identity/v3/projects/255c31b4828b499abeb900572c1ce254 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:07.854227 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de55496-f137-4460-8a88-9219e50c1336 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:07.964894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de55496-f137-4460-8a88-9219e50c1336 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:08.000134 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadbc08e-5031-4483-91ef-7552fbfdde73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:08.001088 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbc08e-5031-4483-91ef-7552fbfdde73 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:08.002143 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbc08e-5031-4483-91ef-7552fbfdde73 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:08.002619 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbc08e-5031-4483-91ef-7552fbfdde73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:08.003643 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadbc08e-5031-4483-91ef-7552fbfdde73 None admin] RBAC: Adding query filter params (role.id=d7a7654a12dd4bd38542203bde537922, scope.project.id=255c31b4828b499abeb900572c1ce254, user.id=e9de1de3860a40fbabbea9687378bcda) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:08.004188 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadbc08e-5031-4483-91ef-7552fbfdde73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:08.005927 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadbc08e-5031-4483-91ef-7552fbfdde73 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:08.012597 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 170/318] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:54:07 2022] GET /identity/v3/role_assignments?user.id=e9de1de3860a40fbabbea9687378bcda&scope.project.id=255c31b4828b499abeb900572c1ce254&role.id=d7a7654a12dd4bd38542203bde537922 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:08.425791 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919f5b6-cc5c-419a-8755-de8ac83f75de None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:08.426306 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919f5b6-cc5c-419a-8755-de8ac83f75de None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:08.426417 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919f5b6-cc5c-419a-8755-de8ac83f75de None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:08.427980 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 149/319] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:08.432996 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d850bc6-c252-40cc-b668-c882124df5c8 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:08.433406 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d850bc6-c252-40cc-b668-c882124df5c8 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:08.433587 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d850bc6-c252-40cc-b668-c882124df5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:08.451522 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-2d850bc6-c252-40cc-b668-c882124df5c8 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:08.452718 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-2d850bc6-c252-40cc-b668-c882124df5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:08.463139 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:08.521829 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 171/320] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:08.544553 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919f5b6-cc5c-419a-8755-de8ac83f75de None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:08.595906 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:08.655491 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919f5b6-cc5c-419a-8755-de8ac83f75de None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:08.691294 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920b479-9bf3-44e5-bdc8-fded21654317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:08.691864 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920b479-9bf3-44e5-bdc8-fded21654317 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:08.692142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920b479-9bf3-44e5-bdc8-fded21654317 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:08.692413 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920b479-9bf3-44e5-bdc8-fded21654317 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:08.693555 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3920b479-9bf3-44e5-bdc8-fded21654317 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:08.695061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3920b479-9bf3-44e5-bdc8-fded21654317 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:08.696970 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 150/321] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:08.702336 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d850bc6-c252-40cc-b668-c882124df5c8 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:08.795332 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15219cb0-3d6f-44d2-8c42-6d65d1a27151 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:08.795623 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15219cb0-3d6f-44d2-8c42-6d65d1a27151 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:08.795891 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15219cb0-3d6f-44d2-8c42-6d65d1a27151 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:08.797248 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 151/322] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:54:08.802135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1c90b-3c82-4f22-adc2-1d94feb267b8 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:08.802426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1c90b-3c82-4f22-adc2-1d94feb267b8 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:08.802704 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1c90b-3c82-4f22-adc2-1d94feb267b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:08.809228 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d850bc6-c252-40cc-b668-c882124df5c8 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:08.822669 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-b9d1c90b-3c82-4f22-adc2-1d94feb267b8 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:08.823984 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b9d1c90b-3c82-4f22-adc2-1d94feb267b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:08.850626 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94aab9-2378-4d1c-9ed3-5f14a5beeb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:08.851289 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94aab9-2378-4d1c-9ed3-5f14a5beeb22 None admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:08.851618 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94aab9-2378-4d1c-9ed3-5f14a5beeb22 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:08.851842 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94aab9-2378-4d1c-9ed3-5f14a5beeb22 None admin] PATH_INFO: `/v3/groups` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:08.852522 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94aab9-2378-4d1c-9ed3-5f14a5beeb22 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:08.853984 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94aab9-2378-4d1c-9ed3-5f14a5beeb22 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:08.895821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 152/323] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 04:54:08.917592 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1c90b-3c82-4f22-adc2-1d94feb267b8 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:08.939287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 172/324] 10.222.0.40 () {60 vars in 1134 bytes} [Wed Aug 3 04:54:08 2022] POST /identity/v3/groups => generated 226 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:54:08.977924 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:09.040953 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1c90b-3c82-4f22-adc2-1d94feb267b8 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:09.078290 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc7df6-2207-41cb-a3bd-ba87d4ae794a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:09.078991 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc7df6-2207-41cb-a3bd-ba87d4ae794a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:09.079279 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc7df6-2207-41cb-a3bd-ba87d4ae794a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:09.080170 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc7df6-2207-41cb-a3bd-ba87d4ae794a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:09.093317 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc7df6-2207-41cb-a3bd-ba87d4ae794a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:09.098295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-cffc7df6-2207-41cb-a3bd-ba87d4ae794a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 04:54:09.098295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:09.098295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:09.098295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:09.098295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:09.098295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:09.098295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:09.098295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:09.098295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:09.098295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:09.098295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:09.098295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:09.098295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:09.098295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:09.098295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:09.098295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:09.099393 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:09.099393 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:09.099393 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:09.099393 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:09.099393 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:09.099393 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:09.099393 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:09.099393 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:09.099393 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:09.099393 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:09.099393 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:09.099393 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:09.099393 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:09.099393 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:09.099393 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:09.099393 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:09.100214 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:09.100214 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:09.100214 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:09.100214 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:09.100214 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:09.100214 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:09.100214 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:09.100214 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:09.100214 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:09.100214 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:09.100214 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 04:54:09.100214 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:09.100214 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 153/325] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:54:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:54:09.112523 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94aab9-2378-4d1c-9ed3-5f14a5beeb22 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:09.116956 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:09.162658 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff04dc-5eed-4fdb-a48d-aa63e76c3c9b None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:09.163020 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff04dc-5eed-4fdb-a48d-aa63e76c3c9b None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:09.163288 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff04dc-5eed-4fdb-a48d-aa63e76c3c9b None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:09.164762 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 154/326] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:54:09.170727 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0055cc-91ed-4187-9140-89d4262db83f None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:09.171041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0055cc-91ed-4187-9140-89d4262db83f None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:09.171315 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0055cc-91ed-4187-9140-89d4262db83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:09.196290 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-1c0055cc-91ed-4187-9140-89d4262db83f None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:09.197630 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1c0055cc-91ed-4187-9140-89d4262db83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:09.239342 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94aab9-2378-4d1c-9ed3-5f14a5beeb22 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:09.279307 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc34c61-866e-43b1-b95f-c8707044c6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:09.280090 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc34c61-866e-43b1-b95f-c8707044c6c6 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:09.280490 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc34c61-866e-43b1-b95f-c8707044c6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:09.280843 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc34c61-866e-43b1-b95f-c8707044c6c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:09.281708 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc34c61-866e-43b1-b95f-c8707044c6c6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:09.282000 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc34c61-866e-43b1-b95f-c8707044c6c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:09.284411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc34c61-866e-43b1-b95f-c8707044c6c6 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:09.292066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 155/327] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 04:54:09.296862 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 173/328] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 3 04:54:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:09.302009 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0055cc-91ed-4187-9140-89d4262db83f None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:09.320351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc34c61-866e-43b1-b95f-c8707044c6c6 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:09.351936 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:09.405952 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:09.429626 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0055cc-91ed-4187-9140-89d4262db83f None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:09.450145 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc34c61-866e-43b1-b95f-c8707044c6c6 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:09.474828 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c9e1e6-da5a-41c5-af27-db9b28269cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:09.475508 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9e1e6-da5a-41c5-af27-db9b28269cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:09.475788 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9e1e6-da5a-41c5-af27-db9b28269cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:09.476062 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9e1e6-da5a-41c5-af27-db9b28269cb9 None admin] PATH_INFO: `/v3/users/99ee0cd592f34497809101d2675b2772` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:09.477362 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c9e1e6-da5a-41c5-af27-db9b28269cb9 None admin] RBAC: Authorizing `identity:get_user(user_id=99ee0cd592f34497809101d2675b2772)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:09.478833 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c9e1e6-da5a-41c5-af27-db9b28269cb9 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:09.481216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 156/329] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:54:09 2022] GET /identity/v3/users/99ee0cd592f34497809101d2675b2772 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:09.487658 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c9e1e6-da5a-41c5-af27-db9b28269cb9 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:09.539542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4e66b-6a29-4b0b-8f81-9dd07c4eafbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:09.540206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4e66b-6a29-4b0b-8f81-9dd07c4eafbf None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:09.540754 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4e66b-6a29-4b0b-8f81-9dd07c4eafbf None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:09.541037 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4e66b-6a29-4b0b-8f81-9dd07c4eafbf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:09.558880 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef4e66b-6a29-4b0b-8f81-9dd07c4eafbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:09.566269 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-bef4e66b-6a29-4b0b-8f81-9dd07c4eafbf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:54:09.566269 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:09.566269 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:09.566269 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:09.566269 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:09.566269 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:09.566269 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:09.566269 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:09.566269 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:09.566269 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:09.566269 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:09.566269 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:09.566269 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:09.566269 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:09.566269 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:09.566269 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:09.567287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:09.567287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:09.567287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:09.567287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:09.567287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:09.567287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:09.567287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:09.567287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:09.567287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:09.567287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:09.567287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:09.567287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:09.567287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:09.567287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:09.567287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:09.567287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:09.568091 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:09.568091 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:09.568091 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:09.568091 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:09.568091 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:09.568091 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:09.568091 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:09.568091 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:09.568091 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:09.568091 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:09.568091 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:54:09.568091 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:09.569243 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 174/330] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:54:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:54:09.576217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4e66b-6a29-4b0b-8f81-9dd07c4eafbf None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:09.653730 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c9e1e6-da5a-41c5-af27-db9b28269cb9 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:09.664949 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:09.703507 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40a70e-881c-48a4-9f0e-66a6181cecf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:09.704173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40a70e-881c-48a4-9f0e-66a6181cecf4 None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:09.704486 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40a70e-881c-48a4-9f0e-66a6181cecf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:09.704765 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40a70e-881c-48a4-9f0e-66a6181cecf4 None admin] PATH_INFO: `/v3/system/users/99ee0cd592f34497809101d2675b2772/roles/d7a7654a12dd4bd38542203bde537922` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:09.706283 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b40a70e-881c-48a4-9f0e-66a6181cecf4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=99ee0cd592f34497809101d2675b2772, role_id=d7a7654a12dd4bd38542203bde537922)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:09.708320 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b40a70e-881c-48a4-9f0e-66a6181cecf4 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:09.742408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4e66b-6a29-4b0b-8f81-9dd07c4eafbf None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:09.773110 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:09.782298 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 157/331] 10.222.0.40 () {58 vars in 1334 bytes} [Wed Aug 3 04:54:09 2022] PUT /identity/v3/system/users/99ee0cd592f34497809101d2675b2772/roles/d7a7654a12dd4bd38542203bde537922 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 04:54:09.783620 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc429f-c494-461f-ac48-9357c4e76b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:09.784452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc429f-c494-461f-ac48-9357c4e76b18 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:09.784828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc429f-c494-461f-ac48-9357c4e76b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:09.785332 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc429f-c494-461f-ac48-9357c4e76b18 None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:09.786228 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfc429f-c494-461f-ac48-9357c4e76b18 None admin] RBAC: Adding query filter params (name=member) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:09.786657 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfc429f-c494-461f-ac48-9357c4e76b18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:09.788554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfc429f-c494-461f-ac48-9357c4e76b18 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:09.800778 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 175/332] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 3 04:54:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:09.806239 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40a70e-881c-48a4-9f0e-66a6181cecf4 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:09.933677 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40a70e-881c-48a4-9f0e-66a6181cecf4 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:09.978057 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96bab24-6d8d-415d-8ee0-f5c5fd289848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:09.978801 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bab24-6d8d-415d-8ee0-f5c5fd289848 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:09.979188 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bab24-6d8d-415d-8ee0-f5c5fd289848 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:09.979469 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bab24-6d8d-415d-8ee0-f5c5fd289848 None admin] PATH_INFO: `/v3/users/95bff512cba344fd9d61ab4a7d324814` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:09.980816 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96bab24-6d8d-415d-8ee0-f5c5fd289848 None admin] RBAC: Authorizing `identity:get_user(user_id=95bff512cba344fd9d61ab4a7d324814)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:09.982383 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96bab24-6d8d-415d-8ee0-f5c5fd289848 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:09.984460 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 158/333] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:54:09 2022] GET /identity/v3/users/95bff512cba344fd9d61ab4a7d324814 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:09.989986 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc429f-c494-461f-ac48-9357c4e76b18 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:10.116140 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc429f-c494-461f-ac48-9357c4e76b18 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:10.164227 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea359c1c-afcf-4adb-8056-0e95f778630d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:10.164903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea359c1c-afcf-4adb-8056-0e95f778630d None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:10.165185 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea359c1c-afcf-4adb-8056-0e95f778630d None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:10.165468 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea359c1c-afcf-4adb-8056-0e95f778630d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:10.166268 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea359c1c-afcf-4adb-8056-0e95f778630d None admin] RBAC: Adding query filter params (role.id=b0e0084bb31348a582616b1421f3b10c, user.id=95bff512cba344fd9d61ab4a7d324814, scope.system=all) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:10.166585 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea359c1c-afcf-4adb-8056-0e95f778630d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:10.168090 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea359c1c-afcf-4adb-8056-0e95f778630d None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:10.182739 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 176/334] 10.222.0.40 () {56 vars in 1308 bytes} [Wed Aug 3 04:54:09 2022] GET /identity/v3/role_assignments?user.id=95bff512cba344fd9d61ab4a7d324814&scope.system=all&role.id=b0e0084bb31348a582616b1421f3b10c => generated 510 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:11.767163 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a271627-c08d-4f60-a454-aea825be8a25 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:11.767921 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a271627-c08d-4f60-a454-aea825be8a25 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:11.768162 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a271627-c08d-4f60-a454-aea825be8a25 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:11.770941 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 159/335] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:11.777418 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19dd29-c7a4-4bea-9b39-c332c3db04db None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:11.778157 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19dd29-c7a4-4bea-9b39-c332c3db04db None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:11.778389 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19dd29-c7a4-4bea-9b39-c332c3db04db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:11.817710 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-9c19dd29-c7a4-4bea-9b39-c332c3db04db None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:11.819406 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-9c19dd29-c7a4-4bea-9b39-c332c3db04db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:11.869099 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:11.901958 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 177/336] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:11.938091 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a271627-c08d-4f60-a454-aea825be8a25 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:11.948783 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a9757-bb32-46d1-984d-0585706da488 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:11.950787 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a9757-bb32-46d1-984d-0585706da488 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:11.951094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a9757-bb32-46d1-984d-0585706da488 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:11.954480 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 178/337] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:11.960041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd248e-147a-4b55-ba63-39c8f56087b0 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:11.960325 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd248e-147a-4b55-ba63-39c8f56087b0 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:11.960610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd248e-147a-4b55-ba63-39c8f56087b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:11.990973 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-07cd248e-147a-4b55-ba63-39c8f56087b0 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:11.992179 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-07cd248e-147a-4b55-ba63-39c8f56087b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:11.994579 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:12.126227 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 179/338] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:12.139545 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a271627-c08d-4f60-a454-aea825be8a25 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:12.167673 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd248e-147a-4b55-ba63-39c8f56087b0 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:12.187298 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d60b7c-9dc0-4b60-9860-4916f9fc3a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:12.187805 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d60b7c-9dc0-4b60-9860-4916f9fc3a98 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:12.188089 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d60b7c-9dc0-4b60-9860-4916f9fc3a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:12.188409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d60b7c-9dc0-4b60-9860-4916f9fc3a98 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:12.212426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d60b7c-9dc0-4b60-9860-4916f9fc3a98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:12.223561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-74d60b7c-9dc0-4b60-9860-4916f9fc3a98 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 04:54:12.223561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:12.223561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:12.223561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:12.223561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:12.223561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:12.223561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:12.223561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:12.223561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:12.223561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:12.223561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:12.223561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:12.223561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:12.223561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:12.223561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:12.223561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:12.224459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:12.224459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:12.224459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:12.224459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:12.224459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:12.224459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:12.224459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:12.224459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:12.224459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:12.224459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:12.224459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:12.224459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:12.224459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:12.224459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:12.224459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:12.224459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:12.225255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:12.225255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:12.225255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:12.225255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:12.225255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:12.225255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:12.225255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:12.225255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:12.225255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:12.225255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:12.225255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 04:54:12.225255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:12.227538 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 160/339] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 3 04:54:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:12.228956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119cbae-1bc4-4247-b1e2-df2143a8f7d9 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:12.229250 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119cbae-1bc4-4247-b1e2-df2143a8f7d9 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:12.229528 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119cbae-1bc4-4247-b1e2-df2143a8f7d9 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:12.231078 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 161/340] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:12.235862 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119cbae-1bc4-4247-b1e2-df2143a8f7d9 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:12.353480 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd248e-147a-4b55-ba63-39c8f56087b0 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:12.366867 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:12.405014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab59e52-07f5-436a-bb2d-38fbccc6f4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:12.405014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab59e52-07f5-436a-bb2d-38fbccc6f4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:12.405014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab59e52-07f5-436a-bb2d-38fbccc6f4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:12.405014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab59e52-07f5-436a-bb2d-38fbccc6f4d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:12.418870 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:12.420440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab59e52-07f5-436a-bb2d-38fbccc6f4d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:12.426901 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-2ab59e52-07f5-436a-bb2d-38fbccc6f4d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:54:12.426901 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:12.426901 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:12.426901 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:12.426901 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:12.426901 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:12.426901 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:12.426901 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:12.426901 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:12.426901 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:12.426901 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:12.426901 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:12.426901 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:12.426901 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:12.426901 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:12.426901 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:12.427868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:12.427868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:12.427868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:12.427868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:12.427868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:12.427868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:12.427868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:12.427868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:12.427868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:12.427868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:12.427868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:12.427868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:12.427868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:12.427868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:12.427868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:12.427868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:12.428674 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:12.428674 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:12.428674 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:12.428674 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:12.428674 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:12.428674 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:12.428674 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:12.428674 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:12.428674 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:12.428674 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:12.428674 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:54:12.428674 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:12.429279 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 180/341] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:54:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:54:12.430249 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d73d37-7aee-4c22-8baf-7f83b3b0f7dd None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:12.430538 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d73d37-7aee-4c22-8baf-7f83b3b0f7dd None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:12.430821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d73d37-7aee-4c22-8baf-7f83b3b0f7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:12.435750 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119cbae-1bc4-4247-b1e2-df2143a8f7d9 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:12.481629 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-55d73d37-7aee-4c22-8baf-7f83b3b0f7dd None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:12.481629 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-55d73d37-7aee-4c22-8baf-7f83b3b0f7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:12.495581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9bfd5b-23ff-4b5c-aac7-c572b4cf0a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:12.496521 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9bfd5b-23ff-4b5c-aac7-c572b4cf0a00 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:12.496815 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9bfd5b-23ff-4b5c-aac7-c572b4cf0a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:12.497101 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9bfd5b-23ff-4b5c-aac7-c572b4cf0a00 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:12.497833 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9bfd5b-23ff-4b5c-aac7-c572b4cf0a00 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:12.498151 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9bfd5b-23ff-4b5c-aac7-c572b4cf0a00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:12.499578 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9bfd5b-23ff-4b5c-aac7-c572b4cf0a00 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:12.512805 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 162/342] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:54:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:12.513533 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9bfd5b-23ff-4b5c-aac7-c572b4cf0a00 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:12.597391 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 181/343] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:12.598643 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d73d37-7aee-4c22-8baf-7f83b3b0f7dd None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:12.704172 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9bfd5b-23ff-4b5c-aac7-c572b4cf0a00 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:12.757307 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:12.773164 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d73d37-7aee-4c22-8baf-7f83b3b0f7dd None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:12.778599 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46619a-1de7-46fb-8016-30fcbe3ceea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:12.779316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46619a-1de7-46fb-8016-30fcbe3ceea4 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:12.779599 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46619a-1de7-46fb-8016-30fcbe3ceea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:12.779876 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46619a-1de7-46fb-8016-30fcbe3ceea4 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:12.780602 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec46619a-1de7-46fb-8016-30fcbe3ceea4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:12.780908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec46619a-1de7-46fb-8016-30fcbe3ceea4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:12.782319 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec46619a-1de7-46fb-8016-30fcbe3ceea4 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:12.805366 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 163/344] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 3 04:54:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:12.806293 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46619a-1de7-46fb-8016-30fcbe3ceea4 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:12.847881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:12.859936 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29f3ea-fbe4-4684-883d-2e730ad5873d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:12.860679 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29f3ea-fbe4-4684-883d-2e730ad5873d None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:12.861146 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29f3ea-fbe4-4684-883d-2e730ad5873d None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:12.861307 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29f3ea-fbe4-4684-883d-2e730ad5873d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:12.984895 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46619a-1de7-46fb-8016-30fcbe3ceea4 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:13.002119 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29f3ea-fbe4-4684-883d-2e730ad5873d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:13.003836 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29f3ea-fbe4-4684-883d-2e730ad5873d None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:13.023648 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf123b33-5881-43bb-bb9b-fa9b1c2cb4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:13.024605 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf123b33-5881-43bb-bb9b-fa9b1c2cb4d0 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:13.025339 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf123b33-5881-43bb-bb9b-fa9b1c2cb4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:13.025808 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf123b33-5881-43bb-bb9b-fa9b1c2cb4d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:13.037683 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf123b33-5881-43bb-bb9b-fa9b1c2cb4d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:13.052067 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-bf123b33-5881-43bb-bb9b-fa9b1c2cb4d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:54:13.052067 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:13.052067 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:13.052067 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:13.052067 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:13.052067 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:13.052067 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:13.052067 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:13.052067 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:13.052067 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:13.052067 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:13.052067 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:13.052067 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:13.052067 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:13.052067 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:13.052067 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:13.053011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:13.053011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:13.053011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:13.053011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:13.053011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:13.053011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:13.053011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:13.053011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:13.053011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:13.053011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:13.053011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:13.053011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:13.053011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:13.053011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:13.053011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:13.053011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:13.053883 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:13.053883 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:13.053883 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:13.053883 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:13.053883 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:13.053883 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:13.053883 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:13.053883 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:13.053883 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:13.053883 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:13.053883 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:54:13.053883 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:13.061279 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 164/345] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:54:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:13.062574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf123b33-5881-43bb-bb9b-fa9b1c2cb4d0 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:13.161890 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:13.170681 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 182/346] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:54:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:54:13.172130 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29f3ea-fbe4-4684-883d-2e730ad5873d None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:13.237331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:13.268984 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf123b33-5881-43bb-bb9b-fa9b1c2cb4d0 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:13.315554 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b47c606-0808-47ad-af84-704677ce9b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:13.316216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47c606-0808-47ad-af84-704677ce9b74 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:13.316502 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47c606-0808-47ad-af84-704677ce9b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:13.316780 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47c606-0808-47ad-af84-704677ce9b74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:13.318872 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29f3ea-fbe4-4684-883d-2e730ad5873d None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:13.363349 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04efe798-4a55-4e8b-b2c5-b556cd02e161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:13.364058 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efe798-4a55-4e8b-b2c5-b556cd02e161 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:13.364347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efe798-4a55-4e8b-b2c5-b556cd02e161 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:13.364735 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efe798-4a55-4e8b-b2c5-b556cd02e161 None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:13.365461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04efe798-4a55-4e8b-b2c5-b556cd02e161 None admin] RBAC: Adding query filter params (name=member) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:13.365772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04efe798-4a55-4e8b-b2c5-b556cd02e161 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:13.367188 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04efe798-4a55-4e8b-b2c5-b556cd02e161 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:13.380055 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 183/347] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 3 04:54:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:13.380978 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04efe798-4a55-4e8b-b2c5-b556cd02e161 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:13.409487 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b47c606-0808-47ad-af84-704677ce9b74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:13.411242 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b47c606-0808-47ad-af84-704677ce9b74 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:13.459365 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:13.513831 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:13.520602 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04efe798-4a55-4e8b-b2c5-b556cd02e161 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:13.544859 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 165/348] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:54:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:54:13.546200 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8abbaf3-79f0-4bd9-acc9-d7b6da9376fe None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:13.547643 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8abbaf3-79f0-4bd9-acc9-d7b6da9376fe None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:13.547643 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8abbaf3-79f0-4bd9-acc9-d7b6da9376fe None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:13.549867 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 166/349] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:13.550690 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8abbaf3-79f0-4bd9-acc9-d7b6da9376fe None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:13.574760 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34fc7c4-b803-438b-9692-24b2bfd4c062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:13.575679 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fc7c4-b803-438b-9692-24b2bfd4c062 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:13.575961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fc7c4-b803-438b-9692-24b2bfd4c062 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:13.576236 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fc7c4-b803-438b-9692-24b2bfd4c062 None admin] PATH_INFO: `/v3/projects/1952481417bf4a2cad0b08d5a7676b61` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:13.577400 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34fc7c4-b803-438b-9692-24b2bfd4c062 None admin] RBAC: Authorizing `identity:get_project(project_id=1952481417bf4a2cad0b08d5a7676b61)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:13.579797 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34fc7c4-b803-438b-9692-24b2bfd4c062 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:13.582726 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 184/350] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:54:13 2022] GET /identity/v3/projects/1952481417bf4a2cad0b08d5a7676b61 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:13.584024 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a32716-8c8e-4597-8245-b7a272343fa6 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:13.584315 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a32716-8c8e-4597-8245-b7a272343fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:13.584593 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a32716-8c8e-4597-8245-b7a272343fa6 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:13.586074 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 185/351] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:13.586686 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a32716-8c8e-4597-8245-b7a272343fa6 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:13.701383 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8abbaf3-79f0-4bd9-acc9-d7b6da9376fe None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:13.737955 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a32716-8c8e-4597-8245-b7a272343fa6 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:13.744432 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7317592e-e094-47fe-b2dc-b6d3c70c786f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:13.745437 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317592e-e094-47fe-b2dc-b6d3c70c786f None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:13.745926 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317592e-e094-47fe-b2dc-b6d3c70c786f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:13.746365 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317592e-e094-47fe-b2dc-b6d3c70c786f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:13.773114 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:13.780969 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc0515-6f35-4a79-a76d-6f076b640f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:13.785451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc0515-6f35-4a79-a76d-6f076b640f88 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:13.785451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc0515-6f35-4a79-a76d-6f076b640f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:13.785451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc0515-6f35-4a79-a76d-6f076b640f88 None admin] PATH_INFO: `/v3/projects/255c31b4828b499abeb900572c1ce254` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:13.785451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc0515-6f35-4a79-a76d-6f076b640f88 None admin] RBAC: Authorizing `identity:get_project(project_id=255c31b4828b499abeb900572c1ce254)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:13.785451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc0515-6f35-4a79-a76d-6f076b640f88 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:13.787264 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 186/352] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:54:13 2022] GET /identity/v3/projects/255c31b4828b499abeb900572c1ce254 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:13.792840 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2024d-2110-45d0-8a67-b7374e43ff4d None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:13.793136 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2024d-2110-45d0-8a67-b7374e43ff4d None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:13.793422 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2024d-2110-45d0-8a67-b7374e43ff4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:13.803476 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:13.815527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-05d2024d-2110-45d0-8a67-b7374e43ff4d None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:13.818526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-05d2024d-2110-45d0-8a67-b7374e43ff4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:13.832032 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7317592e-e094-47fe-b2dc-b6d3c70c786f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:13.834492 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7317592e-e094-47fe-b2dc-b6d3c70c786f None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:13.903942 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 187/353] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:13.903942 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62d3af-7876-423b-85fc-98a9a05a73fc None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:13.903942 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62d3af-7876-423b-85fc-98a9a05a73fc None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:13.903942 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62d3af-7876-423b-85fc-98a9a05a73fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:13.925515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-df62d3af-7876-423b-85fc-98a9a05a73fc None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:13.926713 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-df62d3af-7876-423b-85fc-98a9a05a73fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:13.957366 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 167/354] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:54:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:54:13.958227 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7317592e-e094-47fe-b2dc-b6d3c70c786f None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:14.002485 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 188/355] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:14.003646 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09880271-2306-49bd-ba9e-2df2f41911a2 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:14.003934 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09880271-2306-49bd-ba9e-2df2f41911a2 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:14.004211 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09880271-2306-49bd-ba9e-2df2f41911a2 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:14.006479 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 189/356] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:14.006479 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09880271-2306-49bd-ba9e-2df2f41911a2 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:14.047129 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:14.073631 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7317592e-e094-47fe-b2dc-b6d3c70c786f None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:14.076300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:14.123174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda5ebc-9d7a-442c-b7c0-c5044ecc4343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:14.124071 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda5ebc-9d7a-442c-b7c0-c5044ecc4343 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:14.124402 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda5ebc-9d7a-442c-b7c0-c5044ecc4343 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:14.124660 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda5ebc-9d7a-442c-b7c0-c5044ecc4343 None admin] PATH_INFO: `/v3/groups/131b1e47e2d04feb9e09af36fbbd11ac` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:14.129896 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09880271-2306-49bd-ba9e-2df2f41911a2 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:14.131682 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdda5ebc-9d7a-442c-b7c0-c5044ecc4343 None admin] RBAC: Authorizing `identity:get_group(group_id=131b1e47e2d04feb9e09af36fbbd11ac)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:14.133199 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdda5ebc-9d7a-442c-b7c0-c5044ecc4343 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:14.135103 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 168/357] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Aug 3 04:54:13 2022] GET /identity/v3/groups/131b1e47e2d04feb9e09af36fbbd11ac => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:14.135654 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda5ebc-9d7a-442c-b7c0-c5044ecc4343 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:14.180739 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9c625-0991-4df9-b078-f8d935ff8459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:14.181402 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9c625-0991-4df9-b078-f8d935ff8459 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:14.181690 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9c625-0991-4df9-b078-f8d935ff8459 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:14.182886 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9c625-0991-4df9-b078-f8d935ff8459 None admin] PATH_INFO: `/v3/groups/131b1e47e2d04feb9e09af36fbbd11ac` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:14.183166 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d9c625-0991-4df9-b078-f8d935ff8459 None admin] RBAC: Authorizing `identity:get_group(group_id=131b1e47e2d04feb9e09af36fbbd11ac)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:14.184630 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d9c625-0991-4df9-b078-f8d935ff8459 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:14.186730 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 190/358] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Aug 3 04:54:14 2022] GET /identity/v3/groups/131b1e47e2d04feb9e09af36fbbd11ac => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:14.187061 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9c625-0991-4df9-b078-f8d935ff8459 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:14.249855 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda5ebc-9d7a-442c-b7c0-c5044ecc4343 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:14.300034 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325c637-6178-4014-9bd3-5d7df78a55ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:14.300684 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325c637-6178-4014-9bd3-5d7df78a55ca None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:14.300968 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325c637-6178-4014-9bd3-5d7df78a55ca None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:14.301241 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325c637-6178-4014-9bd3-5d7df78a55ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:14.309593 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9c625-0991-4df9-b078-f8d935ff8459 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:14.313285 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325c637-6178-4014-9bd3-5d7df78a55ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:14.319413 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-4325c637-6178-4014-9bd3-5d7df78a55ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 04:54:14.319413 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:14.319413 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:14.319413 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:14.319413 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:14.319413 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:14.319413 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:14.319413 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:14.319413 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:14.319413 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:14.319413 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:14.319413 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:14.319413 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:14.319413 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:14.319413 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:14.319413 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:14.320567 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:14.320567 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:14.320567 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:14.320567 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:14.320567 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:14.320567 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:14.320567 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:14.320567 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:14.320567 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:14.320567 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:14.320567 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:14.320567 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:14.320567 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:14.320567 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:14.320567 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:14.320567 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:14.321360 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:14.321360 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:14.321360 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:14.321360 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:14.321360 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:14.321360 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:14.321360 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:14.321360 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:14.321360 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:14.321360 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:14.321360 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 04:54:14.321360 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:14.321360 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 169/359] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 3 04:54:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:14.322018 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401c66f-466b-468e-a5ae-602670aae0e4 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:14.322018 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401c66f-466b-468e-a5ae-602670aae0e4 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:14.322322 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401c66f-466b-468e-a5ae-602670aae0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:14.330982 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:14.338999 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:14.344976 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-c401c66f-466b-468e-a5ae-602670aae0e4 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:14.346337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c401c66f-466b-468e-a5ae-602670aae0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:14.351227 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc50be0-e8a4-4898-b6c8-5261acf15421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:14.351893 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc50be0-e8a4-4898-b6c8-5261acf15421 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:14.352174 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc50be0-e8a4-4898-b6c8-5261acf15421 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:14.352449 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc50be0-e8a4-4898-b6c8-5261acf15421 None admin] PATH_INFO: `/v3/projects/1952481417bf4a2cad0b08d5a7676b61` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:14.353649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc50be0-e8a4-4898-b6c8-5261acf15421 None admin] RBAC: Authorizing `identity:get_project(project_id=1952481417bf4a2cad0b08d5a7676b61)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:14.355145 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc50be0-e8a4-4898-b6c8-5261acf15421 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:14.357127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 191/360] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:54:14 2022] GET /identity/v3/projects/1952481417bf4a2cad0b08d5a7676b61 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:14.358003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc50be0-e8a4-4898-b6c8-5261acf15421 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:14.430723 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 170/361] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:14.431392 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401c66f-466b-468e-a5ae-602670aae0e4 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:14.491670 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc50be0-e8a4-4898-b6c8-5261acf15421 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:14.531524 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ee969-da88-47d8-8d31-111a6c4af48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:14.532220 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ee969-da88-47d8-8d31-111a6c4af48a None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:14.532496 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ee969-da88-47d8-8d31-111a6c4af48a None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:14.532775 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ee969-da88-47d8-8d31-111a6c4af48a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:14.543894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ee969-da88-47d8-8d31-111a6c4af48a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:14.550065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-f63ee969-da88-47d8-8d31-111a6c4af48a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 04:54:14.550065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:14.550065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:14.550065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:14.550065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:14.550065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:14.550065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:14.550065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:14.550065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:14.550065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:14.550065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:14.550065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:14.550065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:14.550065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:14.550065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:14.550065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:14.551181 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:14.551181 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:14.551181 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:14.551181 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:14.551181 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:14.551181 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:14.551181 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:14.551181 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:14.551181 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:14.551181 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:14.551181 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:14.551181 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:14.551181 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:14.551181 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:14.551181 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:14.551181 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:14.551991 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:14.551991 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:14.551991 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:14.551991 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:14.551991 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:14.551991 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:14.551991 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:14.551991 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:14.551991 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:14.551991 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:14.551991 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 04:54:14.551991 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:14.551991 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 192/362] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:54:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:14.552661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ee969-da88-47d8-8d31-111a6c4af48a None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:14.553559 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401c66f-466b-468e-a5ae-602670aae0e4 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:14.589542 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:14.595926 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae939a-b54c-4045-80b2-08a86884352e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:14.600419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae939a-b54c-4045-80b2-08a86884352e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:14.600419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae939a-b54c-4045-80b2-08a86884352e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:14.600419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae939a-b54c-4045-80b2-08a86884352e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:14.600419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ae939a-b54c-4045-80b2-08a86884352e None admin] RBAC: Adding query filter params (group.id=131b1e47e2d04feb9e09af36fbbd11ac, role.id=561445e71aff40a0af7b01c539d3cfd9, scope.project.id=1952481417bf4a2cad0b08d5a7676b61) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:14.600419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ae939a-b54c-4045-80b2-08a86884352e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:14.600419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ae939a-b54c-4045-80b2-08a86884352e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:14.605708 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 171/363] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Aug 3 04:54:14 2022] GET /identity/v3/role_assignments?group.id=131b1e47e2d04feb9e09af36fbbd11ac&scope.project.id=1952481417bf4a2cad0b08d5a7676b61&role.id=561445e71aff40a0af7b01c539d3cfd9 => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:14.606336 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae939a-b54c-4045-80b2-08a86884352e None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:14.609590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:14.690316 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ee969-da88-47d8-8d31-111a6c4af48a None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:14.737144 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f3721-f99e-4ff1-a222-ea729f0f80ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:14.737887 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f3721-f99e-4ff1-a222-ea729f0f80ee None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:14.738822 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f3721-f99e-4ff1-a222-ea729f0f80ee None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:14.739192 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f3721-f99e-4ff1-a222-ea729f0f80ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:14.740315 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f3721-f99e-4ff1-a222-ea729f0f80ee None admin] RBAC: Adding query filter params (group.id=131b1e47e2d04feb9e09af36fbbd11ac, role.id=b0e0084bb31348a582616b1421f3b10c, scope.project.id=255c31b4828b499abeb900572c1ce254) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:14.740630 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f3721-f99e-4ff1-a222-ea729f0f80ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:14.742490 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f3721-f99e-4ff1-a222-ea729f0f80ee None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:14.752925 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 193/364] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Aug 3 04:54:14 2022] GET /identity/v3/role_assignments?group.id=131b1e47e2d04feb9e09af36fbbd11ac&scope.project.id=255c31b4828b499abeb900572c1ce254&role.id=b0e0084bb31348a582616b1421f3b10c => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:14.753808 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f3721-f99e-4ff1-a222-ea729f0f80ee None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:14.754722 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae939a-b54c-4045-80b2-08a86884352e None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:14.812500 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4405f97f-1fda-46b1-9708-46d0278c9c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:14.813270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405f97f-1fda-46b1-9708-46d0278c9c24 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:14.813649 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405f97f-1fda-46b1-9708-46d0278c9c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:14.813995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405f97f-1fda-46b1-9708-46d0278c9c24 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:14.814821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4405f97f-1fda-46b1-9708-46d0278c9c24 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:14.815232 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4405f97f-1fda-46b1-9708-46d0278c9c24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:14.816767 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4405f97f-1fda-46b1-9708-46d0278c9c24 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:14.826197 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 172/365] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:54:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:14.828802 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4405f97f-1fda-46b1-9708-46d0278c9c24 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:14.911279 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:14.911907 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f3721-f99e-4ff1-a222-ea729f0f80ee None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:14.936544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:14.965959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4405f97f-1fda-46b1-9708-46d0278c9c24 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:14.970226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee60e28-48b7-4a25-b5f0-761a01ac9f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:14.970924 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee60e28-48b7-4a25-b5f0-761a01ac9f05 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:14.971216 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee60e28-48b7-4a25-b5f0-761a01ac9f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:14.971496 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee60e28-48b7-4a25-b5f0-761a01ac9f05 None admin] PATH_INFO: `/v3/groups/131b1e47e2d04feb9e09af36fbbd11ac` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:14.972687 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee60e28-48b7-4a25-b5f0-761a01ac9f05 None admin] RBAC: Authorizing `identity:get_group(group_id=131b1e47e2d04feb9e09af36fbbd11ac)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:14.974184 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee60e28-48b7-4a25-b5f0-761a01ac9f05 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:14.977758 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 194/366] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Aug 3 04:54:14 2022] GET /identity/v3/groups/131b1e47e2d04feb9e09af36fbbd11ac => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:14.978125 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee60e28-48b7-4a25-b5f0-761a01ac9f05 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:15.014154 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd66de-c4fc-4380-a23c-21f786eb6a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:15.015346 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd66de-c4fc-4380-a23c-21f786eb6a9f None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:15.016082 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd66de-c4fc-4380-a23c-21f786eb6a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:15.016555 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd66de-c4fc-4380-a23c-21f786eb6a9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:15.030662 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dd66de-c4fc-4380-a23c-21f786eb6a9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:15.038731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-33dd66de-c4fc-4380-a23c-21f786eb6a9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:54:15.038731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:15.038731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:15.038731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:15.038731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:15.038731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:15.038731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:15.038731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:15.038731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:15.038731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:15.038731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:15.038731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:15.038731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:15.038731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:15.038731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:15.038731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:15.039908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:15.039908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:15.039908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:15.039908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:15.039908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:15.039908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:15.039908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:15.039908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:15.039908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:15.039908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:15.039908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:15.039908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:15.039908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:15.039908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:15.039908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:15.039908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:15.040908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:15.040908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:15.040908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:15.040908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:15.040908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:15.040908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:15.040908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:15.040908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:15.040908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:15.040908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:15.040908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:54:15.040908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:15.044054 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 173/367] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:54:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:15.045318 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd66de-c4fc-4380-a23c-21f786eb6a9f None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:15.104235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee60e28-48b7-4a25-b5f0-761a01ac9f05 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:15.141547 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0b114-0058-47f8-83ec-4050f708d750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:15.142339 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0b114-0058-47f8-83ec-4050f708d750 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:15.142732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0b114-0058-47f8-83ec-4050f708d750 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:15.143167 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0b114-0058-47f8-83ec-4050f708d750 None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:15.143991 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0b114-0058-47f8-83ec-4050f708d750 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:15.144401 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0b114-0058-47f8-83ec-4050f708d750 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:15.146031 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0b114-0058-47f8-83ec-4050f708d750 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:15.156033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 195/368] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 3 04:54:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:15.157016 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0b114-0058-47f8-83ec-4050f708d750 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:15.166999 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd66de-c4fc-4380-a23c-21f786eb6a9f None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:15.187230 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:15.206468 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc501c-a535-4ce2-a1d1-a50be0dc89b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:15.207433 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc501c-a535-4ce2-a1d1-a50be0dc89b0 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:15.207923 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc501c-a535-4ce2-a1d1-a50be0dc89b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:15.208378 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc501c-a535-4ce2-a1d1-a50be0dc89b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:15.209359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:15.269987 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0b114-0058-47f8-83ec-4050f708d750 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:15.289921 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc501c-a535-4ce2-a1d1-a50be0dc89b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:15.291848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc501c-a535-4ce2-a1d1-a50be0dc89b0 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:15.314403 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b688856-ef81-438a-a9d6-816c93fde70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:15.315175 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b688856-ef81-438a-a9d6-816c93fde70c None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:15.315550 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b688856-ef81-438a-a9d6-816c93fde70c None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:15.315930 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b688856-ef81-438a-a9d6-816c93fde70c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:15.316765 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b688856-ef81-438a-a9d6-816c93fde70c None admin] RBAC: Adding query filter params (group.id=131b1e47e2d04feb9e09af36fbbd11ac, role.id=b0e0084bb31348a582616b1421f3b10c, scope.project.id=1952481417bf4a2cad0b08d5a7676b61) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:15.317152 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b688856-ef81-438a-a9d6-816c93fde70c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:15.318883 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b688856-ef81-438a-a9d6-816c93fde70c None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:15.324662 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 196/369] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Aug 3 04:54:15 2022] GET /identity/v3/role_assignments?group.id=131b1e47e2d04feb9e09af36fbbd11ac&scope.project.id=1952481417bf4a2cad0b08d5a7676b61&role.id=b0e0084bb31348a582616b1421f3b10c => generated 260 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:15.331807 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b688856-ef81-438a-a9d6-816c93fde70c None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:15.452126 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b688856-ef81-438a-a9d6-816c93fde70c None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:15.453377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 174/370] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:54:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:54:15.455011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc501c-a535-4ce2-a1d1-a50be0dc89b0 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:15.473985 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:15.476149 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:15.523672 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac0b66-1c95-4e98-8cbe-112ce0c184a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:15.524238 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac0b66-1c95-4e98-8cbe-112ce0c184a1 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:15.524528 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac0b66-1c95-4e98-8cbe-112ce0c184a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:15.524812 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac0b66-1c95-4e98-8cbe-112ce0c184a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:15.525873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac0b66-1c95-4e98-8cbe-112ce0c184a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:15.526114 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac0b66-1c95-4e98-8cbe-112ce0c184a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:15.527779 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac0b66-1c95-4e98-8cbe-112ce0c184a1 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:15.545726 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 197/371] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:54:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:15.546609 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac0b66-1c95-4e98-8cbe-112ce0c184a1 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:15.569782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc501c-a535-4ce2-a1d1-a50be0dc89b0 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:15.609037 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29677549-e8d7-43cb-9805-a5b3111cc659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:15.610182 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29677549-e8d7-43cb-9805-a5b3111cc659 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:15.610714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29677549-e8d7-43cb-9805-a5b3111cc659 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:15.611217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29677549-e8d7-43cb-9805-a5b3111cc659 None admin] PATH_INFO: `/v3/users/99ee0cd592f34497809101d2675b2772` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:15.612680 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29677549-e8d7-43cb-9805-a5b3111cc659 None admin] RBAC: Authorizing `identity:get_user(user_id=99ee0cd592f34497809101d2675b2772)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:15.614383 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29677549-e8d7-43cb-9805-a5b3111cc659 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:15.616312 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 175/372] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:54:15 2022] GET /identity/v3/users/99ee0cd592f34497809101d2675b2772 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:15.617372 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29677549-e8d7-43cb-9805-a5b3111cc659 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:15.753391 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:15.779520 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29677549-e8d7-43cb-9805-a5b3111cc659 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:15.814099 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac0b66-1c95-4e98-8cbe-112ce0c184a1 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:15.815257 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729aedc-e868-4452-a9eb-0c3418b78449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:15.815880 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729aedc-e868-4452-a9eb-0c3418b78449 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:15.816162 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729aedc-e868-4452-a9eb-0c3418b78449 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:15.816439 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729aedc-e868-4452-a9eb-0c3418b78449 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:15.851739 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bb11f-35c0-42bc-a9fb-8623d46081a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:15.852391 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bb11f-35c0-42bc-a9fb-8623d46081a5 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:15.852677 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bb11f-35c0-42bc-a9fb-8623d46081a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:15.852952 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bb11f-35c0-42bc-a9fb-8623d46081a5 None admin] PATH_INFO: `/v3/projects/255c31b4828b499abeb900572c1ce254` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:15.854337 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94bb11f-35c0-42bc-a9fb-8623d46081a5 None admin] RBAC: Authorizing `identity:get_project(project_id=255c31b4828b499abeb900572c1ce254)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:15.855862 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94bb11f-35c0-42bc-a9fb-8623d46081a5 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:15.857526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 198/373] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:54:15 2022] GET /identity/v3/projects/255c31b4828b499abeb900572c1ce254 => generated 302 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:15.858285 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bb11f-35c0-42bc-a9fb-8623d46081a5 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:15.920755 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:15.922184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5729aedc-e868-4452-a9eb-0c3418b78449 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:15.924268 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5729aedc-e868-4452-a9eb-0c3418b78449 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:16.031963 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bb11f-35c0-42bc-a9fb-8623d46081a5 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:16.097277 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:16.114474 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 176/374] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:54:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:54:16.115643 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729aedc-e868-4452-a9eb-0c3418b78449 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:16.120601 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7246140-baf3-4401-8724-39c22d61e659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:16.121243 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7246140-baf3-4401-8724-39c22d61e659 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:16.121530 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7246140-baf3-4401-8724-39c22d61e659 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:16.121809 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7246140-baf3-4401-8724-39c22d61e659 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:16.122610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7246140-baf3-4401-8724-39c22d61e659 None admin] RBAC: Adding query filter params (role.id=d7a7654a12dd4bd38542203bde537922, user.id=99ee0cd592f34497809101d2675b2772, scope.system=all) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:16.122949 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7246140-baf3-4401-8724-39c22d61e659 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:16.124408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7246140-baf3-4401-8724-39c22d61e659 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:16.158353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 199/375] 10.222.0.40 () {56 vars in 1308 bytes} [Wed Aug 3 04:54:15 2022] GET /identity/v3/role_assignments?user.id=99ee0cd592f34497809101d2675b2772&scope.system=all&role.id=d7a7654a12dd4bd38542203bde537922 => generated 510 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:16.159236 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7246140-baf3-4401-8724-39c22d61e659 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:16.227492 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729aedc-e868-4452-a9eb-0c3418b78449 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:16.273043 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d870d-8992-4141-bdc1-4b631e2b7162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:16.274120 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d870d-8992-4141-bdc1-4b631e2b7162 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:16.274608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d870d-8992-4141-bdc1-4b631e2b7162 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:16.275375 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d870d-8992-4141-bdc1-4b631e2b7162 None admin] PATH_INFO: `/v3/groups/131b1e47e2d04feb9e09af36fbbd11ac` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:16.279683 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d870d-8992-4141-bdc1-4b631e2b7162 None admin] RBAC: Authorizing `identity:get_group(group_id=131b1e47e2d04feb9e09af36fbbd11ac)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:16.281478 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d870d-8992-4141-bdc1-4b631e2b7162 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:16.284475 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 177/376] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Aug 3 04:54:16 2022] GET /identity/v3/groups/131b1e47e2d04feb9e09af36fbbd11ac => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:16.290961 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d870d-8992-4141-bdc1-4b631e2b7162 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:16.386984 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:16.417403 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:16.433858 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7246140-baf3-4401-8724-39c22d61e659 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:16.441220 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d870d-8992-4141-bdc1-4b631e2b7162 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:16.482816 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bb685-7413-4f0d-bf45-24d6d3c3f724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:16.483798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bb685-7413-4f0d-bf45-24d6d3c3f724 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:16.484282 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bb685-7413-4f0d-bf45-24d6d3c3f724 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:16.484757 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bb685-7413-4f0d-bf45-24d6d3c3f724 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:16.487629 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882bb685-7413-4f0d-bf45-24d6d3c3f724 None admin] RBAC: Adding query filter params (group.id=131b1e47e2d04feb9e09af36fbbd11ac, role.id=561445e71aff40a0af7b01c539d3cfd9, scope.project.id=255c31b4828b499abeb900572c1ce254) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:16.488202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882bb685-7413-4f0d-bf45-24d6d3c3f724 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:16.489854 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882bb685-7413-4f0d-bf45-24d6d3c3f724 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:16.500807 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 178/377] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Aug 3 04:54:16 2022] GET /identity/v3/role_assignments?group.id=131b1e47e2d04feb9e09af36fbbd11ac&scope.project.id=255c31b4828b499abeb900572c1ce254&role.id=561445e71aff40a0af7b01c539d3cfd9 => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:16.509694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9f896-d53b-420c-8aa2-32fd19f20505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:16.510389 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9f896-d53b-420c-8aa2-32fd19f20505 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:16.510678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9f896-d53b-420c-8aa2-32fd19f20505 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:16.511022 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9f896-d53b-420c-8aa2-32fd19f20505 None admin] PATH_INFO: `/v3/projects/6917683e025f412781fd852422d27ce7` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:16.512325 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9f896-d53b-420c-8aa2-32fd19f20505 None admin] RBAC: Authorizing `identity:get_project(project_id=6917683e025f412781fd852422d27ce7)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:16.513834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9f896-d53b-420c-8aa2-32fd19f20505 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:16.515884 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 200/378] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:54:16 2022] GET /identity/v3/projects/6917683e025f412781fd852422d27ce7 => generated 344 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:16.521990 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bb685-7413-4f0d-bf45-24d6d3c3f724 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:16.635255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bb685-7413-4f0d-bf45-24d6d3c3f724 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:16.654906 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:16.688338 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99baddc-bf88-42f2-9f88-925cf1c1b8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:16.689442 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99baddc-bf88-42f2-9f88-925cf1c1b8bb None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:16.689916 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99baddc-bf88-42f2-9f88-925cf1c1b8bb None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:16.690389 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99baddc-bf88-42f2-9f88-925cf1c1b8bb None admin] PATH_INFO: `/v3/groups/90fa737b91fd4eaabcfc7a1814d2221c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:16.700930 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99baddc-bf88-42f2-9f88-925cf1c1b8bb None admin] RBAC: Authorizing `identity:get_group(group_id=90fa737b91fd4eaabcfc7a1814d2221c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:16.702666 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99baddc-bf88-42f2-9f88-925cf1c1b8bb None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:16.705623 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 179/379] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Aug 3 04:54:16 2022] GET /identity/v3/groups/90fa737b91fd4eaabcfc7a1814d2221c => generated 229 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:16.712634 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9f896-d53b-420c-8aa2-32fd19f20505 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:16.871816 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9f896-d53b-420c-8aa2-32fd19f20505 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:16.915803 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c0a9d-3a08-4cf3-b34e-971c295ae595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:16.916760 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c0a9d-3a08-4cf3-b34e-971c295ae595 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:16.917256 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c0a9d-3a08-4cf3-b34e-971c295ae595 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:16.917720 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c0a9d-3a08-4cf3-b34e-971c295ae595 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:16.918722 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c0a9d-3a08-4cf3-b34e-971c295ae595 None admin] RBAC: Adding query filter params (group.id=90fa737b91fd4eaabcfc7a1814d2221c, role.id=e39d4ecea4c64d5794e1079c9be3ad36, scope.project.id=6917683e025f412781fd852422d27ce7) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:16.919250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c0a9d-3a08-4cf3-b34e-971c295ae595 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:16.920936 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c0a9d-3a08-4cf3-b34e-971c295ae595 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:16.928061 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 201/380] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Aug 3 04:54:16 2022] GET /identity/v3/role_assignments?group.id=90fa737b91fd4eaabcfc7a1814d2221c&scope.project.id=6917683e025f412781fd852422d27ce7&role.id=e39d4ecea4c64d5794e1079c9be3ad36 => generated 260 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:17.100201 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a55c4a-dc5b-4055-bf9e-4514f33d2bf3 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:17.100710 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a55c4a-dc5b-4055-bf9e-4514f33d2bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:17.100783 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a55c4a-dc5b-4055-bf9e-4514f33d2bf3 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:17.102380 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 180/381] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:17.108665 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb96cb7-5f4f-4aae-bbea-1c5c81208510 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:17.109323 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb96cb7-5f4f-4aae-bbea-1c5c81208510 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:17.109843 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb96cb7-5f4f-4aae-bbea-1c5c81208510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:17.131907 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-2bb96cb7-5f4f-4aae-bbea-1c5c81208510 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:17.133486 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-2bb96cb7-5f4f-4aae-bbea-1c5c81208510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:17.170103 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:17.229288 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 202/382] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:17.265842 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a55c4a-dc5b-4055-bf9e-4514f33d2bf3 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:17.317735 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e2138-dc21-48d3-9540-797707b6fe5d None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:17.318052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e2138-dc21-48d3-9540-797707b6fe5d None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:17.318298 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e2138-dc21-48d3-9540-797707b6fe5d None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:17.320604 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 203/383] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:54:17.327500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ee9da-f474-43ff-b557-defd95926b47 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:17.327789 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ee9da-f474-43ff-b557-defd95926b47 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:17.328069 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ee9da-f474-43ff-b557-defd95926b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:17.361035 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-ab1ee9da-f474-43ff-b557-defd95926b47 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:17.362383 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-ab1ee9da-f474-43ff-b557-defd95926b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:17.421736 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a55c4a-dc5b-4055-bf9e-4514f33d2bf3 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:17.470726 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e75c7c-9de9-4be7-9236-a0e3d1ff1cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:17.471872 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e75c7c-9de9-4be7-9236-a0e3d1ff1cad None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:17.472361 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e75c7c-9de9-4be7-9236-a0e3d1ff1cad None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:17.472819 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e75c7c-9de9-4be7-9236-a0e3d1ff1cad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:17.496920 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e75c7c-9de9-4be7-9236-a0e3d1ff1cad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:17.503040 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-c6e75c7c-9de9-4be7-9236-a0e3d1ff1cad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 04:54:17.503040 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:17.503040 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:17.503040 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:17.503040 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:17.503040 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:17.503040 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:17.503040 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:17.503040 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:17.503040 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:17.503040 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:17.503040 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:17.503040 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:17.503040 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:17.503040 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:17.503040 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:17.503992 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:17.503992 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:17.503992 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:17.503992 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:17.503992 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:17.503992 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:17.503992 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:17.503992 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:17.503992 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:17.503992 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:17.503992 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:17.503992 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:17.503992 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:17.503992 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:17.503992 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:17.503992 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:17.505235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:17.505235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:17.505235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:17.505235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:17.505235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:17.505235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:17.505235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:17.505235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:17.505235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:17.505235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:17.505235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 04:54:17.505235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:17.505235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 181/384] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 3 04:54:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:17.512042 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e75c7c-9de9-4be7-9236-a0e3d1ff1cad None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:17.536924 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 204/385] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 04:54:17.571628 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ee9da-f474-43ff-b557-defd95926b47 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:17.583971 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:17.693231 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e75c7c-9de9-4be7-9236-a0e3d1ff1cad None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:17.731952 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685670b-91d0-43b9-8470-b75406c8d7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:17.732756 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685670b-91d0-43b9-8470-b75406c8d7e5 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:17.733045 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685670b-91d0-43b9-8470-b75406c8d7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:17.733323 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685670b-91d0-43b9-8470-b75406c8d7e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:17.734043 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685670b-91d0-43b9-8470-b75406c8d7e5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:17.734349 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685670b-91d0-43b9-8470-b75406c8d7e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:17.736161 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685670b-91d0-43b9-8470-b75406c8d7e5 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:17.745327 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 182/386] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:54:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:17.750692 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685670b-91d0-43b9-8470-b75406c8d7e5 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:17.834250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:17.864949 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685670b-91d0-43b9-8470-b75406c8d7e5 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:17.876593 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:17.884077 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ee9da-f474-43ff-b557-defd95926b47 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:17.903636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c7951-cf63-4ac5-bf4d-0a0c8e1d6950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:17.904513 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c7951-cf63-4ac5-bf4d-0a0c8e1d6950 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:17.904967 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c7951-cf63-4ac5-bf4d-0a0c8e1d6950 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:17.905443 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c7951-cf63-4ac5-bf4d-0a0c8e1d6950 None admin] PATH_INFO: `/v3/groups/131b1e47e2d04feb9e09af36fbbd11ac` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:17.906760 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82c7951-cf63-4ac5-bf4d-0a0c8e1d6950 None admin] RBAC: Authorizing `identity:get_group(group_id=131b1e47e2d04feb9e09af36fbbd11ac)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:17.908487 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82c7951-cf63-4ac5-bf4d-0a0c8e1d6950 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:17.910528 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 183/387] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Aug 3 04:54:17 2022] GET /identity/v3/groups/131b1e47e2d04feb9e09af36fbbd11ac => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:17.920002 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c7951-cf63-4ac5-bf4d-0a0c8e1d6950 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:17.957037 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd0cc9-73bc-4817-b92e-c10f2c60a58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:17.958711 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd0cc9-73bc-4817-b92e-c10f2c60a58d None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:17.959189 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd0cc9-73bc-4817-b92e-c10f2c60a58d None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:17.960751 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd0cc9-73bc-4817-b92e-c10f2c60a58d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:17.984754 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd0cc9-73bc-4817-b92e-c10f2c60a58d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:17.994033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-9edd0cc9-73bc-4817-b92e-c10f2c60a58d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:54:17.994033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:17.994033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:17.994033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:17.994033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:17.994033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:17.994033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:17.994033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:17.994033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:17.994033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:17.994033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:17.994033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:17.994033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:17.994033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:17.994033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:17.994033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:17.995229 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:17.995229 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:17.995229 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:17.995229 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:17.995229 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:17.995229 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:17.995229 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:17.995229 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:17.995229 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:17.995229 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:17.995229 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:17.995229 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:17.995229 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:17.995229 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:17.995229 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:17.995229 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:17.996226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:17.996226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:17.996226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:17.996226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:17.996226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:17.996226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:17.996226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:17.996226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:17.996226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:17.996226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:17.996226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:54:17.996226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:18.005451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 205/388] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:54:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 434 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:18.010780 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd0cc9-73bc-4817-b92e-c10f2c60a58d None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:18.109804 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c7951-cf63-4ac5-bf4d-0a0c8e1d6950 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:18.146876 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd0cc9-73bc-4817-b92e-c10f2c60a58d None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:18.156106 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86768859-6141-4c59-9114-bc4f10c64955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:18.157409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86768859-6141-4c59-9114-bc4f10c64955 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:18.157841 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86768859-6141-4c59-9114-bc4f10c64955 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:18.158285 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86768859-6141-4c59-9114-bc4f10c64955 None admin] PATH_INFO: `/v3/projects/1952481417bf4a2cad0b08d5a7676b61` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:18.160427 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86768859-6141-4c59-9114-bc4f10c64955 None admin] RBAC: Authorizing `identity:get_project(project_id=1952481417bf4a2cad0b08d5a7676b61)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:18.162719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86768859-6141-4c59-9114-bc4f10c64955 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:18.165075 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 184/389] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:54:17 2022] GET /identity/v3/projects/1952481417bf4a2cad0b08d5a7676b61 => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:18.166717 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573519d-f512-420f-9071-9cfd552cb12d None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:18.167175 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573519d-f512-420f-9071-9cfd552cb12d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:18.167607 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573519d-f512-420f-9071-9cfd552cb12d None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:18.168951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 185/390] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:18.170779 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573519d-f512-420f-9071-9cfd552cb12d None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:18.190092 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27c08eb-e88b-42f9-8027-f3a8a3f667d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:18.190867 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c08eb-e88b-42f9-8027-f3a8a3f667d8 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:18.191399 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c08eb-e88b-42f9-8027-f3a8a3f667d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:18.191779 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c08eb-e88b-42f9-8027-f3a8a3f667d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:18.192724 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27c08eb-e88b-42f9-8027-f3a8a3f667d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:18.193093 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27c08eb-e88b-42f9-8027-f3a8a3f667d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:18.194584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27c08eb-e88b-42f9-8027-f3a8a3f667d8 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:18.204034 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 206/391] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 3 04:54:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:18.205444 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14333656-0e59-4fa2-b18a-20cb1a63f275 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:18.205842 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14333656-0e59-4fa2-b18a-20cb1a63f275 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:18.206216 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14333656-0e59-4fa2-b18a-20cb1a63f275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:18.225055 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:18.225795 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-14333656-0e59-4fa2-b18a-20cb1a63f275 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:18.227649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-14333656-0e59-4fa2-b18a-20cb1a63f275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:18.231738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:18.299912 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 207/392] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:18.301867 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573519d-f512-420f-9071-9cfd552cb12d None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:18.307791 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14333656-0e59-4fa2-b18a-20cb1a63f275 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:18.343313 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea11f940-0d99-4148-8947-c570a8e697de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:18.345013 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11f940-0d99-4148-8947-c570a8e697de None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:18.345767 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11f940-0d99-4148-8947-c570a8e697de None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:18.346261 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11f940-0d99-4148-8947-c570a8e697de None admin] PATH_INFO: `/v3/projects/1952481417bf4a2cad0b08d5a7676b61/groups/131b1e47e2d04feb9e09af36fbbd11ac/roles/561445e71aff40a0af7b01c539d3cfd9` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:18.350604 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea11f940-0d99-4148-8947-c570a8e697de None admin] RBAC: Authorizing `identity:create_grant(project_id=1952481417bf4a2cad0b08d5a7676b61, group_id=131b1e47e2d04feb9e09af36fbbd11ac, role_id=561445e71aff40a0af7b01c539d3cfd9)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:18.351946 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea11f940-0d99-4148-8947-c570a8e697de None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:18.428097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 186/393] 10.222.0.40 () {58 vars in 1442 bytes} [Wed Aug 3 04:54:18 2022] PUT /identity/v3/projects/1952481417bf4a2cad0b08d5a7676b61/groups/131b1e47e2d04feb9e09af36fbbd11ac/roles/561445e71aff40a0af7b01c539d3cfd9 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 04:54:18.429698 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea11f940-0d99-4148-8947-c570a8e697de None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:18.458424 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14333656-0e59-4fa2-b18a-20cb1a63f275 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:18.539126 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d2b06-f3b5-4567-838c-5069fe959df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:18.540669 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d2b06-f3b5-4567-838c-5069fe959df9 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:18.540669 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d2b06-f3b5-4567-838c-5069fe959df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:18.540669 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d2b06-f3b5-4567-838c-5069fe959df9 None admin] PATH_INFO: `/v3/groups/131b1e47e2d04feb9e09af36fbbd11ac` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:18.541962 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d2b06-f3b5-4567-838c-5069fe959df9 None admin] RBAC: Authorizing `identity:get_group(group_id=131b1e47e2d04feb9e09af36fbbd11ac)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:18.543708 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d2b06-f3b5-4567-838c-5069fe959df9 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:18.545771 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 208/394] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Aug 3 04:54:18 2022] GET /identity/v3/groups/131b1e47e2d04feb9e09af36fbbd11ac => generated 226 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:18.558870 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d2b06-f3b5-4567-838c-5069fe959df9 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:18.575320 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:18.627627 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:18.632162 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea11f940-0d99-4148-8947-c570a8e697de None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:18.677040 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcf36f-b2da-46fc-9cd1-2713796da31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:18.677626 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcf36f-b2da-46fc-9cd1-2713796da31a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:18.677910 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcf36f-b2da-46fc-9cd1-2713796da31a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:18.678189 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcf36f-b2da-46fc-9cd1-2713796da31a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:18.690688 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bcf36f-b2da-46fc-9cd1-2713796da31a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:18.695821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-23bcf36f-b2da-46fc-9cd1-2713796da31a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:54:18.695821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:18.695821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:18.695821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:18.695821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:18.695821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:18.695821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:18.695821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:18.695821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:18.695821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:18.695821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:18.695821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:18.695821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:18.695821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:18.695821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:18.695821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:18.696714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:18.696714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:18.696714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:18.696714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:18.696714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:18.696714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:18.696714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:18.696714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:18.696714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:18.696714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:18.696714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:18.696714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:18.696714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:18.696714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:18.696714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:18.696714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:18.697507 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:18.697507 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:18.697507 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:18.697507 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:18.697507 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:18.697507 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:18.697507 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:18.697507 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:18.697507 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:18.697507 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:18.697507 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:54:18.697507 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:18.697507 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 187/395] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:54:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:18.703318 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcf36f-b2da-46fc-9cd1-2713796da31a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:18.746568 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d2b06-f3b5-4567-838c-5069fe959df9 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:18.790045 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd793d65-3744-4d18-b31f-5a138aaa4716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:18.790610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd793d65-3744-4d18-b31f-5a138aaa4716 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:18.790925 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd793d65-3744-4d18-b31f-5a138aaa4716 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:18.791209 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd793d65-3744-4d18-b31f-5a138aaa4716 None admin] PATH_INFO: `/v3/projects/255c31b4828b499abeb900572c1ce254` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:18.792466 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd793d65-3744-4d18-b31f-5a138aaa4716 None admin] RBAC: Authorizing `identity:get_project(project_id=255c31b4828b499abeb900572c1ce254)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:18.793989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd793d65-3744-4d18-b31f-5a138aaa4716 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:18.795855 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 209/396] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:54:18 2022] GET /identity/v3/projects/255c31b4828b499abeb900572c1ce254 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:18.801961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd793d65-3744-4d18-b31f-5a138aaa4716 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:18.830963 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcf36f-b2da-46fc-9cd1-2713796da31a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:18.889049 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:18.903660 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac801d-ba7b-40ac-900d-7663ea35293a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:18.904422 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac801d-ba7b-40ac-900d-7663ea35293a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:18.904699 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac801d-ba7b-40ac-900d-7663ea35293a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:18.905058 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac801d-ba7b-40ac-900d-7663ea35293a None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:18.905908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ac801d-ba7b-40ac-900d-7663ea35293a None admin] RBAC: Adding query filter params (name=member) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:18.906273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ac801d-ba7b-40ac-900d-7663ea35293a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:18.908476 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ac801d-ba7b-40ac-900d-7663ea35293a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:18.917259 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd793d65-3744-4d18-b31f-5a138aaa4716 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:18.925661 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 188/397] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 3 04:54:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:18.931252 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac801d-ba7b-40ac-900d-7663ea35293a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:18.960326 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f142b46-751d-47ac-bd95-70067c1343fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:18.962209 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f142b46-751d-47ac-bd95-70067c1343fb None admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:18.962815 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f142b46-751d-47ac-bd95-70067c1343fb None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:18.963425 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f142b46-751d-47ac-bd95-70067c1343fb None admin] PATH_INFO: `/v3/projects/255c31b4828b499abeb900572c1ce254/groups/131b1e47e2d04feb9e09af36fbbd11ac/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:18.965801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f142b46-751d-47ac-bd95-70067c1343fb None admin] RBAC: Authorizing `identity:create_grant(project_id=255c31b4828b499abeb900572c1ce254, group_id=131b1e47e2d04feb9e09af36fbbd11ac, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:18.967921 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f142b46-751d-47ac-bd95-70067c1343fb None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:18.976203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:19.044083 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 210/398] 10.222.0.40 () {58 vars in 1442 bytes} [Wed Aug 3 04:54:18 2022] PUT /identity/v3/projects/255c31b4828b499abeb900572c1ce254/groups/131b1e47e2d04feb9e09af36fbbd11ac/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 04:54:19.045663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a503c-da8f-424f-9d1b-738e5376e117 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:19.046150 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a503c-da8f-424f-9d1b-738e5376e117 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:19.046625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a503c-da8f-424f-9d1b-738e5376e117 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:19.049253 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 211/399] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:19.054768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3bedc-b942-48ad-ad90-2d914bc33c1e None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:19.055108 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3bedc-b942-48ad-ad90-2d914bc33c1e None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:19.055343 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3bedc-b942-48ad-ad90-2d914bc33c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:19.071030 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac801d-ba7b-40ac-900d-7663ea35293a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:19.076672 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-89c3bedc-b942-48ad-ad90-2d914bc33c1e None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:19.077887 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-89c3bedc-b942-48ad-ad90-2d914bc33c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:19.149652 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec1221-1762-4656-80c4-fd8e1a174f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:19.150593 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec1221-1762-4656-80c4-fd8e1a174f82 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:19.150872 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec1221-1762-4656-80c4-fd8e1a174f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:19.151157 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec1221-1762-4656-80c4-fd8e1a174f82 None admin] PATH_INFO: `/v3/groups/131b1e47e2d04feb9e09af36fbbd11ac` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:19.156025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ec1221-1762-4656-80c4-fd8e1a174f82 None admin] RBAC: Authorizing `identity:get_group(group_id=131b1e47e2d04feb9e09af36fbbd11ac)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:19.159124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ec1221-1762-4656-80c4-fd8e1a174f82 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:19.163679 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 189/400] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Aug 3 04:54:18 2022] GET /identity/v3/groups/131b1e47e2d04feb9e09af36fbbd11ac => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:19.169348 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec1221-1762-4656-80c4-fd8e1a174f82 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:19.229189 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 212/401] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:19.264302 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3bedc-b942-48ad-ad90-2d914bc33c1e None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:19.275793 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:19.286228 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec1221-1762-4656-80c4-fd8e1a174f82 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:19.323806 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca5ee2-3cf9-4908-800b-fe74807fab2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:19.324425 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca5ee2-3cf9-4908-800b-fe74807fab2d None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:19.324715 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca5ee2-3cf9-4908-800b-fe74807fab2d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:19.324983 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca5ee2-3cf9-4908-800b-fe74807fab2d None admin] PATH_INFO: `/v3/projects/1952481417bf4a2cad0b08d5a7676b61` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:19.326211 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca5ee2-3cf9-4908-800b-fe74807fab2d None admin] RBAC: Authorizing `identity:get_project(project_id=1952481417bf4a2cad0b08d5a7676b61)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:19.328115 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca5ee2-3cf9-4908-800b-fe74807fab2d None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:19.330302 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 190/402] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:54:19 2022] GET /identity/v3/projects/1952481417bf4a2cad0b08d5a7676b61 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:19.335979 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca5ee2-3cf9-4908-800b-fe74807fab2d None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:19.344272 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:19.389668 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3bedc-b942-48ad-ad90-2d914bc33c1e None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:19.435553 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e020fb-b092-4e80-a4dd-cb5071014044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:19.436205 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e020fb-b092-4e80-a4dd-cb5071014044 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:19.436487 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e020fb-b092-4e80-a4dd-cb5071014044 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:19.436759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e020fb-b092-4e80-a4dd-cb5071014044 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:19.451318 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e020fb-b092-4e80-a4dd-cb5071014044 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:19.451837 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca5ee2-3cf9-4908-800b-fe74807fab2d None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:19.457932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-58e020fb-b092-4e80-a4dd-cb5071014044 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 04:54:19.457932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:19.457932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:19.457932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:19.457932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:19.457932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:19.457932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:19.457932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:19.457932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:19.457932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:19.457932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:19.457932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:19.457932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:19.457932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:19.457932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:19.457932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:19.458834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:19.458834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:19.458834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:19.458834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:19.458834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:19.458834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:19.458834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:19.458834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:19.458834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:19.458834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:19.458834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:19.458834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:19.458834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:19.458834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:19.458834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:19.458834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:19.459688 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:19.459688 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:19.459688 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:19.459688 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:19.459688 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:19.459688 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:19.459688 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:19.459688 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:19.459688 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:19.459688 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:19.459688 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 04:54:19.459688 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:19.459688 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 213/403] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 3 04:54:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:54:19.465212 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29555ec-a9de-4d7e-bfdb-454e7ba6a218 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:19.465212 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29555ec-a9de-4d7e-bfdb-454e7ba6a218 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:19.465212 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29555ec-a9de-4d7e-bfdb-454e7ba6a218 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:19.465212 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 214/404] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:19.467208 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29555ec-a9de-4d7e-bfdb-454e7ba6a218 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:19.505200 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b9555d-9195-43ec-886e-5d96b878c154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:19.505880 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9555d-9195-43ec-886e-5d96b878c154 None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:19.506158 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9555d-9195-43ec-886e-5d96b878c154 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:19.506436 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9555d-9195-43ec-886e-5d96b878c154 None admin] PATH_INFO: `/v3/projects/1952481417bf4a2cad0b08d5a7676b61/groups/131b1e47e2d04feb9e09af36fbbd11ac/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:19.508485 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b9555d-9195-43ec-886e-5d96b878c154 None admin] RBAC: Authorizing `identity:create_grant(project_id=1952481417bf4a2cad0b08d5a7676b61, group_id=131b1e47e2d04feb9e09af36fbbd11ac, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:19.511021 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b9555d-9195-43ec-886e-5d96b878c154 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:19.583309 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29555ec-a9de-4d7e-bfdb-454e7ba6a218 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:19.596125 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 191/405] 10.222.0.40 () {58 vars in 1442 bytes} [Wed Aug 3 04:54:19 2022] PUT /identity/v3/projects/1952481417bf4a2cad0b08d5a7676b61/groups/131b1e47e2d04feb9e09af36fbbd11ac/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 04:54:19.598475 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc8ddf-09f6-469a-b1c2-f40bc3235921 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:19.599016 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc8ddf-09f6-469a-b1c2-f40bc3235921 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:19.599986 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc8ddf-09f6-469a-b1c2-f40bc3235921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:19.614899 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:19.628057 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-3dfc8ddf-09f6-469a-b1c2-f40bc3235921 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:19.630315 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3dfc8ddf-09f6-469a-b1c2-f40bc3235921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:19.647459 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc4217-2023-4e6c-ac55-c696c3f0159c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:19.648117 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc4217-2023-4e6c-ac55-c696c3f0159c None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:19.648555 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc4217-2023-4e6c-ac55-c696c3f0159c None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:19.648766 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc4217-2023-4e6c-ac55-c696c3f0159c None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:19.650638 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc4217-2023-4e6c-ac55-c696c3f0159c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:19.650992 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc4217-2023-4e6c-ac55-c696c3f0159c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:19.654807 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc4217-2023-4e6c-ac55-c696c3f0159c None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:19.678462 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:19.694585 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 215/406] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:54:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:19.700479 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc4217-2023-4e6c-ac55-c696c3f0159c None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:19.782713 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 192/407] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:19.812550 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc8ddf-09f6-469a-b1c2-f40bc3235921 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:19.841782 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc4217-2023-4e6c-ac55-c696c3f0159c None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:19.891516 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fdfbe0-ae65-45f4-b4d6-f36925c1001f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:19.892183 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fdfbe0-ae65-45f4-b4d6-f36925c1001f None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:19.892462 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fdfbe0-ae65-45f4-b4d6-f36925c1001f None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:19.892736 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fdfbe0-ae65-45f4-b4d6-f36925c1001f None admin] PATH_INFO: `/v3/groups/131b1e47e2d04feb9e09af36fbbd11ac` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:19.894607 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fdfbe0-ae65-45f4-b4d6-f36925c1001f None admin] RBAC: Authorizing `identity:get_group(group_id=131b1e47e2d04feb9e09af36fbbd11ac)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:19.896094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fdfbe0-ae65-45f4-b4d6-f36925c1001f None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:19.897723 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 216/408] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Aug 3 04:54:19 2022] GET /identity/v3/groups/131b1e47e2d04feb9e09af36fbbd11ac => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:19.905595 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fdfbe0-ae65-45f4-b4d6-f36925c1001f None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:19.964176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc8ddf-09f6-469a-b1c2-f40bc3235921 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:20.023200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:20.036051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21871695-9ab2-47ad-be9a-dfbba74fd7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:20.038999 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21871695-9ab2-47ad-be9a-dfbba74fd7a3 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:20.038999 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21871695-9ab2-47ad-be9a-dfbba74fd7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:20.038999 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21871695-9ab2-47ad-be9a-dfbba74fd7a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:20.062396 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21871695-9ab2-47ad-be9a-dfbba74fd7a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:20.067589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-21871695-9ab2-47ad-be9a-dfbba74fd7a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:54:20.067589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:20.067589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:20.067589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:20.067589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:20.067589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:20.067589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:20.067589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:20.067589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:20.067589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:20.067589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:20.067589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:20.067589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:20.067589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:20.067589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:20.067589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:20.068498 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:20.068498 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:20.068498 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:20.068498 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:20.068498 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:20.068498 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:20.068498 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:20.068498 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:20.068498 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:20.068498 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:20.068498 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:20.068498 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:20.068498 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:20.068498 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:20.068498 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:20.068498 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:20.069331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:20.069331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:20.069331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:20.069331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:20.069331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:20.069331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:20.069331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:20.069331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:20.069331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:20.069331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:20.069331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:54:20.069331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:20.078251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 193/409] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:54:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:54:20.079967 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21871695-9ab2-47ad-be9a-dfbba74fd7a3 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:20.093883 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:20.124978 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fdfbe0-ae65-45f4-b4d6-f36925c1001f None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:20.198665 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21871695-9ab2-47ad-be9a-dfbba74fd7a3 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:20.220094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667a196-b2a6-44b1-b44b-169e798ac1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:20.220763 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667a196-b2a6-44b1-b44b-169e798ac1e6 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:20.221044 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667a196-b2a6-44b1-b44b-169e798ac1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:20.221321 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667a196-b2a6-44b1-b44b-169e798ac1e6 None admin] PATH_INFO: `/v3/projects/255c31b4828b499abeb900572c1ce254` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:20.222556 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667a196-b2a6-44b1-b44b-169e798ac1e6 None admin] RBAC: Authorizing `identity:get_project(project_id=255c31b4828b499abeb900572c1ce254)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:20.224021 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667a196-b2a6-44b1-b44b-169e798ac1e6 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:20.226046 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 217/410] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:54:19 2022] GET /identity/v3/projects/255c31b4828b499abeb900572c1ce254 => generated 302 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:20.232742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667a196-b2a6-44b1-b44b-169e798ac1e6 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:20.241790 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8573e-30f8-4bf8-b527-e620ab810eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:20.242469 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8573e-30f8-4bf8-b527-e620ab810eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:20.242755 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8573e-30f8-4bf8-b527-e620ab810eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:20.243051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8573e-30f8-4bf8-b527-e620ab810eb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:20.244753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8573e-30f8-4bf8-b527-e620ab810eb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:20.244989 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8573e-30f8-4bf8-b527-e620ab810eb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:20.246365 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8573e-30f8-4bf8-b527-e620ab810eb9 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:20.259730 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 194/411] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:54:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:20.264286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8573e-30f8-4bf8-b527-e620ab810eb9 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:20.371265 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:20.405680 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8573e-30f8-4bf8-b527-e620ab810eb9 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:20.438786 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667a196-b2a6-44b1-b44b-169e798ac1e6 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:20.463274 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:20.467806 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6029eb64-41a8-471c-8b9e-5c3d71fffe01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:20.468778 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6029eb64-41a8-471c-8b9e-5c3d71fffe01 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:20.469381 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6029eb64-41a8-471c-8b9e-5c3d71fffe01 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:20.469858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6029eb64-41a8-471c-8b9e-5c3d71fffe01 None admin] PATH_INFO: `/v3/groups/90fa737b91fd4eaabcfc7a1814d2221c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:20.473698 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6029eb64-41a8-471c-8b9e-5c3d71fffe01 None admin] RBAC: Authorizing `identity:get_group(group_id=90fa737b91fd4eaabcfc7a1814d2221c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:20.475833 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef207b06-f4ee-44c0-bc92-c03298dd770e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:20.475833 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef207b06-f4ee-44c0-bc92-c03298dd770e None admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:20.475833 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef207b06-f4ee-44c0-bc92-c03298dd770e None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:20.476066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6029eb64-41a8-471c-8b9e-5c3d71fffe01 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:20.476135 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef207b06-f4ee-44c0-bc92-c03298dd770e None admin] PATH_INFO: `/v3/projects/255c31b4828b499abeb900572c1ce254/groups/131b1e47e2d04feb9e09af36fbbd11ac/roles/561445e71aff40a0af7b01c539d3cfd9` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:20.477250 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 195/412] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Aug 3 04:54:20 2022] GET /identity/v3/groups/90fa737b91fd4eaabcfc7a1814d2221c => generated 229 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:20.477815 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef207b06-f4ee-44c0-bc92-c03298dd770e None admin] RBAC: Authorizing `identity:create_grant(project_id=255c31b4828b499abeb900572c1ce254, group_id=131b1e47e2d04feb9e09af36fbbd11ac, role_id=561445e71aff40a0af7b01c539d3cfd9)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:20.479437 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef207b06-f4ee-44c0-bc92-c03298dd770e None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:20.482418 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6029eb64-41a8-471c-8b9e-5c3d71fffe01 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:20.547156 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 218/413] 10.222.0.40 () {58 vars in 1442 bytes} [Wed Aug 3 04:54:20 2022] PUT /identity/v3/projects/255c31b4828b499abeb900572c1ce254/groups/131b1e47e2d04feb9e09af36fbbd11ac/roles/561445e71aff40a0af7b01c539d3cfd9 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 04:54:20.629098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6029eb64-41a8-471c-8b9e-5c3d71fffe01 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:20.683662 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edf7592-b23c-42d3-ac4b-8f7201387906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:20.684328 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf7592-b23c-42d3-ac4b-8f7201387906 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:20.684618 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf7592-b23c-42d3-ac4b-8f7201387906 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:20.684902 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf7592-b23c-42d3-ac4b-8f7201387906 None admin] PATH_INFO: `/v3/projects/6917683e025f412781fd852422d27ce7` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:20.687068 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edf7592-b23c-42d3-ac4b-8f7201387906 None admin] RBAC: Authorizing `identity:get_project(project_id=6917683e025f412781fd852422d27ce7)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:20.689235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edf7592-b23c-42d3-ac4b-8f7201387906 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:20.691361 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 196/414] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:54:20 2022] GET /identity/v3/projects/6917683e025f412781fd852422d27ce7 => generated 344 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:20.697080 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef207b06-f4ee-44c0-bc92-c03298dd770e None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:20.846124 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef207b06-f4ee-44c0-bc92-c03298dd770e None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:20.881001 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378dff3-54d3-49f6-b8a4-d17a59057629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:20.881594 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378dff3-54d3-49f6-b8a4-d17a59057629 None admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:20.881875 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378dff3-54d3-49f6-b8a4-d17a59057629 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:20.882245 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378dff3-54d3-49f6-b8a4-d17a59057629 None admin] PATH_INFO: `/v3/projects/6917683e025f412781fd852422d27ce7/groups/90fa737b91fd4eaabcfc7a1814d2221c/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:20.884215 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e378dff3-54d3-49f6-b8a4-d17a59057629 None admin] RBAC: Authorizing `identity:create_grant(project_id=6917683e025f412781fd852422d27ce7, group_id=90fa737b91fd4eaabcfc7a1814d2221c, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:20.885653 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e378dff3-54d3-49f6-b8a4-d17a59057629 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:20.905271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06d482-2177-465e-865f-4e8ad1bf1f25 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:20.905555 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06d482-2177-465e-865f-4e8ad1bf1f25 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:20.905847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06d482-2177-465e-865f-4e8ad1bf1f25 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:20.907769 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 197/415] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:54:20.912791 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56b600-60f3-410f-a112-acdd4eb6ddce None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:20.913083 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56b600-60f3-410f-a112-acdd4eb6ddce None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:20.913367 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56b600-60f3-410f-a112-acdd4eb6ddce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:20.927158 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:20.933598 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-fb56b600-60f3-410f-a112-acdd4eb6ddce None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:20.935115 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-fb56b600-60f3-410f-a112-acdd4eb6ddce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:20.967600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 219/416] 10.222.0.40 () {58 vars in 1442 bytes} [Wed Aug 3 04:54:20 2022] PUT /identity/v3/projects/6917683e025f412781fd852422d27ce7/groups/90fa737b91fd4eaabcfc7a1814d2221c/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 04:54:21.043898 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 198/417] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 04:54:21.089344 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378dff3-54d3-49f6-b8a4-d17a59057629 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:21.155742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:21.225946 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378dff3-54d3-49f6-b8a4-d17a59057629 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:21.263838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1771fe-ddc5-44a4-88aa-87e78d208ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:21.264503 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1771fe-ddc5-44a4-88aa-87e78d208ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:21.264790 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1771fe-ddc5-44a4-88aa-87e78d208ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:21.265067 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1771fe-ddc5-44a4-88aa-87e78d208ec5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:21.276385 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1771fe-ddc5-44a4-88aa-87e78d208ec5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:21.281967 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-9c1771fe-ddc5-44a4-88aa-87e78d208ec5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 04:54:21.281967 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:21.281967 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:21.281967 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:21.281967 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:21.281967 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:21.281967 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:21.281967 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:21.281967 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:21.281967 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:21.281967 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:21.281967 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:21.281967 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:21.281967 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:21.281967 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:21.281967 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:21.282875 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:21.282875 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:21.282875 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:21.282875 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:21.282875 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:21.282875 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:21.282875 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:21.282875 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:21.282875 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:21.282875 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:21.282875 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:21.282875 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:21.282875 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:21.282875 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:21.282875 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:21.282875 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:21.283685 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:21.283685 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:21.283685 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:21.283685 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:21.283685 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:21.283685 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:21.283685 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:21.283685 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:21.283685 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:21.283685 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:21.283685 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 04:54:21.283685 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:21.283685 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 220/418] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 3 04:54:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:21.289035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56b600-60f3-410f-a112-acdd4eb6ddce None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:21.431521 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56b600-60f3-410f-a112-acdd4eb6ddce None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:21.475229 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ca84e-27e2-4e37-a706-f5ceab3f8731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:21.475880 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ca84e-27e2-4e37-a706-f5ceab3f8731 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:21.476164 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ca84e-27e2-4e37-a706-f5ceab3f8731 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:21.476436 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ca84e-27e2-4e37-a706-f5ceab3f8731 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:21.477165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ca84e-27e2-4e37-a706-f5ceab3f8731 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:21.477525 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ca84e-27e2-4e37-a706-f5ceab3f8731 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:21.478940 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ca84e-27e2-4e37-a706-f5ceab3f8731 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:21.491394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:21.494794 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 199/419] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:54:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:21.500432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1771fe-ddc5-44a4-88aa-87e78d208ec5 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:21.656130 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1771fe-ddc5-44a4-88aa-87e78d208ec5 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:21.667954 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:21.708206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef164a4-9fce-441f-96a3-d4b08beac437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:21.708755 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef164a4-9fce-441f-96a3-d4b08beac437 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:21.709042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef164a4-9fce-441f-96a3-d4b08beac437 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:21.709325 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef164a4-9fce-441f-96a3-d4b08beac437 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:21.781446 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf0900-7cf4-4399-b414-e603d23aa740 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:21.782282 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf0900-7cf4-4399-b414-e603d23aa740 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:21.782535 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf0900-7cf4-4399-b414-e603d23aa740 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:21.788259 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 200/420] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:21.790110 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db2c07-6a71-4e14-9043-8ec2e2c7f2ec None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:21.790472 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db2c07-6a71-4e14-9043-8ec2e2c7f2ec None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:21.790859 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db2c07-6a71-4e14-9043-8ec2e2c7f2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:21.803733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef164a4-9fce-441f-96a3-d4b08beac437 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:21.805975 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef164a4-9fce-441f-96a3-d4b08beac437 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:21.812240 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-c5db2c07-6a71-4e14-9043-8ec2e2c7f2ec None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:21.813784 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c5db2c07-6a71-4e14-9043-8ec2e2c7f2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:21.946319 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 201/421] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 04:54:21.983768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 221/422] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:54:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:54:21.986226 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db2c07-6a71-4e14-9043-8ec2e2c7f2ec None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:21.993619 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef164a4-9fce-441f-96a3-d4b08beac437 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:21.997886 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:22.096900 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef164a4-9fce-441f-96a3-d4b08beac437 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:22.136413 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8b6b2-b30e-4182-a25d-4a17d65771c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:22.137425 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8b6b2-b30e-4182-a25d-4a17d65771c9 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:22.137901 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8b6b2-b30e-4182-a25d-4a17d65771c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:22.138363 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8b6b2-b30e-4182-a25d-4a17d65771c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:22.150162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b8b6b2-b30e-4182-a25d-4a17d65771c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:22.155544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-f2b8b6b2-b30e-4182-a25d-4a17d65771c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:54:22.155544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:22.155544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:22.155544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:22.155544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:22.155544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:22.155544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:22.155544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:22.155544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:22.155544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:22.155544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:22.155544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:22.155544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:22.155544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:22.155544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:22.155544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:22.156440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:22.156440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:22.156440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:22.156440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:22.156440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:22.156440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:22.156440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:22.156440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:22.156440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:22.156440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:22.156440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:22.156440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:22.156440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:22.156440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:22.156440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:22.156440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:22.157294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:22.157294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:22.157294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:22.157294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:22.157294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:22.157294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:22.157294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:22.157294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:22.157294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:22.157294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:22.157294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:54:22.157294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:22.165024 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 222/423] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:54:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:54:22.167377 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8b6b2-b30e-4182-a25d-4a17d65771c9 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:22.214190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db2c07-6a71-4e14-9043-8ec2e2c7f2ec None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:22.245020 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:22.253792 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f337f1e-b07f-4ecb-b4e7-ee0471446176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:22.267097 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:22.267217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f337f1e-b07f-4ecb-b4e7-ee0471446176 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:22.267217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f337f1e-b07f-4ecb-b4e7-ee0471446176 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:22.267217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f337f1e-b07f-4ecb-b4e7-ee0471446176 None admin] PATH_INFO: `/v3/projects/1952481417bf4a2cad0b08d5a7676b61` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:22.267217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f337f1e-b07f-4ecb-b4e7-ee0471446176 None admin] RBAC: Authorizing `identity:get_project(project_id=1952481417bf4a2cad0b08d5a7676b61)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:22.267217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f337f1e-b07f-4ecb-b4e7-ee0471446176 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:22.267217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 202/424] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:54:21 2022] GET /identity/v3/projects/1952481417bf4a2cad0b08d5a7676b61 => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:22.267568 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06842e01-efff-4c3c-8099-40feaacadc33 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:22.267568 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06842e01-efff-4c3c-8099-40feaacadc33 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:22.267568 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06842e01-efff-4c3c-8099-40feaacadc33 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:22.267568 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 203/425] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:22.268857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06842e01-efff-4c3c-8099-40feaacadc33 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:22.301305 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8b6b2-b30e-4182-a25d-4a17d65771c9 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:22.347669 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f3f8fd-c441-4497-ac99-041f6016327b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:22.348369 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3f8fd-c441-4497-ac99-041f6016327b None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:22.348665 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3f8fd-c441-4497-ac99-041f6016327b None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:22.349008 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3f8fd-c441-4497-ac99-041f6016327b None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:22.349846 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f3f8fd-c441-4497-ac99-041f6016327b None admin] RBAC: Adding query filter params (name=member) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:22.350161 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f3f8fd-c441-4497-ac99-041f6016327b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:22.352534 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f3f8fd-c441-4497-ac99-041f6016327b None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:22.364576 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 223/426] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 3 04:54:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:22.366940 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a9377-ab5b-4aa3-9a40-3b22c6502b69 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:22.367461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a9377-ab5b-4aa3-9a40-3b22c6502b69 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:22.367941 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a9377-ab5b-4aa3-9a40-3b22c6502b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:22.386583 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06842e01-efff-4c3c-8099-40feaacadc33 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:22.393548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-669a9377-ab5b-4aa3-9a40-3b22c6502b69 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:22.395355 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-669a9377-ab5b-4aa3-9a40-3b22c6502b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:22.425874 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e529ce-a0c5-4d78-8679-791fae6e6b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:22.426906 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e529ce-a0c5-4d78-8679-791fae6e6b0e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:22.427398 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e529ce-a0c5-4d78-8679-791fae6e6b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:22.427954 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e529ce-a0c5-4d78-8679-791fae6e6b0e None admin] PATH_INFO: `/v3/groups/131b1e47e2d04feb9e09af36fbbd11ac` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:22.429492 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e529ce-a0c5-4d78-8679-791fae6e6b0e None admin] RBAC: Authorizing `identity:get_group(group_id=131b1e47e2d04feb9e09af36fbbd11ac)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:22.431383 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e529ce-a0c5-4d78-8679-791fae6e6b0e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:22.433597 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 204/427] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Aug 3 04:54:22 2022] GET /identity/v3/groups/131b1e47e2d04feb9e09af36fbbd11ac => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:22.434801 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e529ce-a0c5-4d78-8679-791fae6e6b0e None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:22.498958 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 224/428] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:22.498958 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669a9377-ab5b-4aa3-9a40-3b22c6502b69 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:22.543205 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:22.570766 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:22.588596 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e529ce-a0c5-4d78-8679-791fae6e6b0e None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:22.627943 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43811c-1f09-465a-96ea-88d01ed2819a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:22.628613 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43811c-1f09-465a-96ea-88d01ed2819a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:22.628894 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43811c-1f09-465a-96ea-88d01ed2819a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:22.629220 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43811c-1f09-465a-96ea-88d01ed2819a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:22.634097 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669a9377-ab5b-4aa3-9a40-3b22c6502b69 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:22.679590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b617338-91b9-4bf2-874a-6ff9492e43b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:22.680333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b617338-91b9-4bf2-874a-6ff9492e43b4 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:22.680733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b617338-91b9-4bf2-874a-6ff9492e43b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:22.681106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b617338-91b9-4bf2-874a-6ff9492e43b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:22.681992 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b617338-91b9-4bf2-874a-6ff9492e43b4 None admin] RBAC: Adding query filter params (group.id=131b1e47e2d04feb9e09af36fbbd11ac, role.id=561445e71aff40a0af7b01c539d3cfd9, scope.project.id=1952481417bf4a2cad0b08d5a7676b61) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:22.682401 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b617338-91b9-4bf2-874a-6ff9492e43b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:22.683959 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b617338-91b9-4bf2-874a-6ff9492e43b4 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:22.690239 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 225/429] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Aug 3 04:54:22 2022] GET /identity/v3/role_assignments?group.id=131b1e47e2d04feb9e09af36fbbd11ac&scope.project.id=1952481417bf4a2cad0b08d5a7676b61&role.id=561445e71aff40a0af7b01c539d3cfd9 => generated 611 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:22.691489 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b617338-91b9-4bf2-874a-6ff9492e43b4 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:22.728077 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43811c-1f09-465a-96ea-88d01ed2819a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:22.730140 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43811c-1f09-465a-96ea-88d01ed2819a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:22.806585 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b617338-91b9-4bf2-874a-6ff9492e43b4 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:22.822088 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:22.854523 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:22.872705 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 205/430] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:54:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:54:22.886102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43811c-1f09-465a-96ea-88d01ed2819a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:22.905061 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9754921-d7de-4d3e-b804-700a3f95498d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:22.908104 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9754921-d7de-4d3e-b804-700a3f95498d None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:22.908397 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9754921-d7de-4d3e-b804-700a3f95498d None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:22.908676 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9754921-d7de-4d3e-b804-700a3f95498d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:22.943827 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9754921-d7de-4d3e-b804-700a3f95498d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:22.950357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-b9754921-d7de-4d3e-b804-700a3f95498d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:54:22.950357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:22.950357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:22.950357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:22.950357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:22.950357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:22.950357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:22.950357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:22.950357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:22.950357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:22.950357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:22.950357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:22.950357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:22.950357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:22.950357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:22.950357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:22.951558 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:22.951558 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:22.951558 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:22.951558 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:22.951558 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:22.951558 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:22.951558 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:22.951558 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:22.951558 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:22.951558 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:22.951558 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:22.951558 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:22.951558 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:22.951558 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:22.951558 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:22.951558 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:22.952596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:22.952596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:22.952596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:22.952596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:22.952596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:22.952596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:22.952596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:22.952596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:22.952596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:22.952596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:22.952596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 04:54:22.952596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:22.952596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 226/431] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:54:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:22.960337 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9754921-d7de-4d3e-b804-700a3f95498d None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:23.013296 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43811c-1f09-465a-96ea-88d01ed2819a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:23.060783 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eded318-c1ac-458a-ab45-27c6f6167d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:23.061676 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eded318-c1ac-458a-ab45-27c6f6167d9a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:23.062127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eded318-c1ac-458a-ab45-27c6f6167d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:23.062409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eded318-c1ac-458a-ab45-27c6f6167d9a None admin] PATH_INFO: `/v3/projects/255c31b4828b499abeb900572c1ce254` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:23.064186 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eded318-c1ac-458a-ab45-27c6f6167d9a None admin] RBAC: Authorizing `identity:get_project(project_id=255c31b4828b499abeb900572c1ce254)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:23.065649 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eded318-c1ac-458a-ab45-27c6f6167d9a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:23.070955 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 206/432] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:54:22 2022] GET /identity/v3/projects/255c31b4828b499abeb900572c1ce254 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:23.070955 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ea34c-6db2-4ccd-959c-522307c2c511 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:23.070955 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ea34c-6db2-4ccd-959c-522307c2c511 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:23.070955 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ea34c-6db2-4ccd-959c-522307c2c511 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:23.070955 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 207/433] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:23.075566 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ea34c-6db2-4ccd-959c-522307c2c511 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:23.085615 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9754921-d7de-4d3e-b804-700a3f95498d None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:23.108179 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:23.122788 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ac61b-db8c-4d1a-ac6c-975376142a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:23.140987 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ac61b-db8c-4d1a-ac6c-975376142a47 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:23.140987 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ac61b-db8c-4d1a-ac6c-975376142a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:23.140987 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ac61b-db8c-4d1a-ac6c-975376142a47 None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:23.140987 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ac61b-db8c-4d1a-ac6c-975376142a47 None admin] RBAC: Adding query filter params (name=member) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:23.140987 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ac61b-db8c-4d1a-ac6c-975376142a47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:23.140987 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ac61b-db8c-4d1a-ac6c-975376142a47 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:23.142024 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 227/434] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 3 04:54:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:23.142024 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a06eb-4e3b-47f5-b56c-a0de0fde6ce0 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:23.142024 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a06eb-4e3b-47f5-b56c-a0de0fde6ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:23.142024 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a06eb-4e3b-47f5-b56c-a0de0fde6ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:23.158485 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-ea3a06eb-4e3b-47f5-b56c-a0de0fde6ce0 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:23.159741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-ea3a06eb-4e3b-47f5-b56c-a0de0fde6ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:23.190641 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ea34c-6db2-4ccd-959c-522307c2c511 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:23.196326 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:23.228180 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd12e9-3818-49ae-8629-35b7a46bf8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:23.228813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd12e9-3818-49ae-8629-35b7a46bf8bc None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:23.229094 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd12e9-3818-49ae-8629-35b7a46bf8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:23.229365 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd12e9-3818-49ae-8629-35b7a46bf8bc None admin] PATH_INFO: `/v3/groups/131b1e47e2d04feb9e09af36fbbd11ac` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:23.230949 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 228/435] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:23.230949 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a06eb-4e3b-47f5-b56c-a0de0fde6ce0 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:23.231970 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bd12e9-3818-49ae-8629-35b7a46bf8bc None admin] RBAC: Authorizing `identity:get_group(group_id=131b1e47e2d04feb9e09af36fbbd11ac)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:23.233402 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bd12e9-3818-49ae-8629-35b7a46bf8bc None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:23.239853 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 208/436] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Aug 3 04:54:23 2022] GET /identity/v3/groups/131b1e47e2d04feb9e09af36fbbd11ac => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:23.240668 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd12e9-3818-49ae-8629-35b7a46bf8bc None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:23.347189 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a06eb-4e3b-47f5-b56c-a0de0fde6ce0 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:23.350755 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:23.353489 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd12e9-3818-49ae-8629-35b7a46bf8bc None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:23.384255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b48e3-2f11-4a56-9df7-dbac9278fea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:23.384905 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b48e3-2f11-4a56-9df7-dbac9278fea1 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:23.385160 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b48e3-2f11-4a56-9df7-dbac9278fea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:23.385437 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b48e3-2f11-4a56-9df7-dbac9278fea1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:23.392165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bb64a-58d8-4d48-82f0-3971971a96a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:23.392782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bb64a-58d8-4d48-82f0-3971971a96a7 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:23.393062 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bb64a-58d8-4d48-82f0-3971971a96a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:23.393333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bb64a-58d8-4d48-82f0-3971971a96a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:23.394502 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0bb64a-58d8-4d48-82f0-3971971a96a7 None admin] RBAC: Adding query filter params (group.id=131b1e47e2d04feb9e09af36fbbd11ac, role.id=b0e0084bb31348a582616b1421f3b10c, scope.project.id=255c31b4828b499abeb900572c1ce254) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:23.394829 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0bb64a-58d8-4d48-82f0-3971971a96a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:23.396287 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0bb64a-58d8-4d48-82f0-3971971a96a7 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:23.410897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 209/437] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Aug 3 04:54:23 2022] GET /identity/v3/role_assignments?group.id=131b1e47e2d04feb9e09af36fbbd11ac&scope.project.id=255c31b4828b499abeb900572c1ce254&role.id=b0e0084bb31348a582616b1421f3b10c => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:23.410897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bb64a-58d8-4d48-82f0-3971971a96a7 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:23.445171 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:23.476316 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599b48e3-2f11-4a56-9df7-dbac9278fea1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:23.477908 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599b48e3-2f11-4a56-9df7-dbac9278fea1 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:23.544399 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bb64a-58d8-4d48-82f0-3971971a96a7 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:23.585173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a2ad3-fad5-485c-90f6-3234aafd904f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:23.585827 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a2ad3-fad5-485c-90f6-3234aafd904f None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:23.586111 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a2ad3-fad5-485c-90f6-3234aafd904f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:23.586881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a2ad3-fad5-485c-90f6-3234aafd904f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:23.599163 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a2ad3-fad5-485c-90f6-3234aafd904f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:23.600076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 229/438] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:54:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:54:23.601487 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a407ed-fd1c-43aa-bd66-0eb4dd675ac8 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:23.601935 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a407ed-fd1c-43aa-bd66-0eb4dd675ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:23.602356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a407ed-fd1c-43aa-bd66-0eb4dd675ac8 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:23.603879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 230/439] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:23.604404 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-c80a2ad3-fad5-485c-90f6-3234aafd904f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 04:54:23.604404 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:23.604404 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:23.604404 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:23.604404 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:23.604404 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:23.604404 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:23.604404 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:23.604404 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:23.604404 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:23.604404 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:23.604404 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:23.604404 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:23.604404 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:23.604404 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:23.604404 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:23.605249 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:23.605249 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:23.605249 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:23.605249 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:23.605249 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:23.605249 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:23.605249 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:23.605249 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:23.605249 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:23.605249 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:23.605249 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:23.605249 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:23.605249 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:23.605249 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:23.605249 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:23.605249 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:23.606024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:23.606024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:23.606024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:23.606024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:23.606024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:23.606024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:23.606024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:23.606024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:23.606024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:23.606024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:23.606024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 04:54:23.606024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:23.607565 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a407ed-fd1c-43aa-bd66-0eb4dd675ac8 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:23.610986 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 210/440] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 3 04:54:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:23.613396 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de277505-ffbb-4f92-948b-f4b22b3cc62a None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:23.613690 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de277505-ffbb-4f92-948b-f4b22b3cc62a None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:23.613971 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de277505-ffbb-4f92-948b-f4b22b3cc62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:23.635417 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-de277505-ffbb-4f92-948b-f4b22b3cc62a None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:23.636602 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-de277505-ffbb-4f92-948b-f4b22b3cc62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:23.655091 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:23.683293 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:23.716483 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a407ed-fd1c-43aa-bd66-0eb4dd675ac8 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:23.719466 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 211/441] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:23.720146 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de277505-ffbb-4f92-948b-f4b22b3cc62a None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:23.756678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11e400-bf2b-41cc-8c40-c38cf78dd3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:23.757600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11e400-bf2b-41cc-8c40-c38cf78dd3dc None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:23.757600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11e400-bf2b-41cc-8c40-c38cf78dd3dc None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:23.757873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11e400-bf2b-41cc-8c40-c38cf78dd3dc None admin] PATH_INFO: `/v3/projects/1952481417bf4a2cad0b08d5a7676b61` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:23.759106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae11e400-bf2b-41cc-8c40-c38cf78dd3dc None admin] RBAC: Authorizing `identity:get_project(project_id=1952481417bf4a2cad0b08d5a7676b61)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:23.760582 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae11e400-bf2b-41cc-8c40-c38cf78dd3dc None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:23.762245 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 231/442] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:54:23 2022] GET /identity/v3/projects/1952481417bf4a2cad0b08d5a7676b61 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:23.764046 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11e400-bf2b-41cc-8c40-c38cf78dd3dc None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:23.828254 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de277505-ffbb-4f92-948b-f4b22b3cc62a None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:23.864422 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11e400-bf2b-41cc-8c40-c38cf78dd3dc None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:23.867726 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43be5d-fe26-45b5-92b5-5ba07d464144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:23.868350 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43be5d-fe26-45b5-92b5-5ba07d464144 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:23.868634 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43be5d-fe26-45b5-92b5-5ba07d464144 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:23.868911 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43be5d-fe26-45b5-92b5-5ba07d464144 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:23.869630 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c43be5d-fe26-45b5-92b5-5ba07d464144 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:23.869941 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c43be5d-fe26-45b5-92b5-5ba07d464144 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:23.871344 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c43be5d-fe26-45b5-92b5-5ba07d464144 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:23.879828 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 212/443] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:54:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:23.880627 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43be5d-fe26-45b5-92b5-5ba07d464144 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:23.899080 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:23.899237 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3310ec-4fda-430d-8248-2ea9175d6461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:23.900066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3310ec-4fda-430d-8248-2ea9175d6461 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:23.900357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3310ec-4fda-430d-8248-2ea9175d6461 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:23.900632 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3310ec-4fda-430d-8248-2ea9175d6461 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:23.909351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:23.912456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3310ec-4fda-430d-8248-2ea9175d6461 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:23.917584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-eb3310ec-4fda-430d-8248-2ea9175d6461 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:54:23.917584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:23.917584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:23.917584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:23.917584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:23.917584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:23.917584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:23.917584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:23.917584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:23.917584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:23.917584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:23.917584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:23.917584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:23.917584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:23.917584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:23.917584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:23.918675 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:23.918675 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:23.918675 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:23.918675 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:23.918675 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:23.918675 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:23.918675 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:23.918675 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:23.918675 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:23.918675 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:23.918675 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:23.918675 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:23.918675 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:23.918675 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:23.918675 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:23.918675 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:23.919839 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:23.919839 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:23.919839 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:23.919839 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:23.919839 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:23.919839 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:23.919839 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:23.919839 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:23.919839 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:23.919839 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:23.919839 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:54:23.919839 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:23.919839 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 232/444] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:54:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:23.919839 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3310ec-4fda-430d-8248-2ea9175d6461 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:23.983950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43be5d-fe26-45b5-92b5-5ba07d464144 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:24.018121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379b9f9-95b2-4f5b-92dd-6aed516a80b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:24.018747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379b9f9-95b2-4f5b-92dd-6aed516a80b3 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:24.018879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3310ec-4fda-430d-8248-2ea9175d6461 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:24.019058 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379b9f9-95b2-4f5b-92dd-6aed516a80b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:24.019327 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379b9f9-95b2-4f5b-92dd-6aed516a80b3 None admin] PATH_INFO: `/v3/groups/131b1e47e2d04feb9e09af36fbbd11ac` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:24.020457 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379b9f9-95b2-4f5b-92dd-6aed516a80b3 None admin] RBAC: Authorizing `identity:get_group(group_id=131b1e47e2d04feb9e09af36fbbd11ac)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:24.021888 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379b9f9-95b2-4f5b-92dd-6aed516a80b3 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:24.023585 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 213/445] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Aug 3 04:54:23 2022] GET /identity/v3/groups/131b1e47e2d04feb9e09af36fbbd11ac => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:24.024338 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379b9f9-95b2-4f5b-92dd-6aed516a80b3 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:24.055956 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadb3af-6fa6-47a2-87e5-93e541ff0ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:24.056572 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadb3af-6fa6-47a2-87e5-93e541ff0ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:24.056856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadb3af-6fa6-47a2-87e5-93e541ff0ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:24.057130 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadb3af-6fa6-47a2-87e5-93e541ff0ed2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:24.109001 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:24.136950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:24.136950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aadb3af-6fa6-47a2-87e5-93e541ff0ed2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:24.137532 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379b9f9-95b2-4f5b-92dd-6aed516a80b3 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:24.137626 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aadb3af-6fa6-47a2-87e5-93e541ff0ed2 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:24.159177 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c1cc0-f339-48bb-a7ba-08fc0f4c4317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:24.160847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c1cc0-f339-48bb-a7ba-08fc0f4c4317 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:24.160847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c1cc0-f339-48bb-a7ba-08fc0f4c4317 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:24.160847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c1cc0-f339-48bb-a7ba-08fc0f4c4317 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:24.161099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c1cc0-f339-48bb-a7ba-08fc0f4c4317 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:24.161316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c1cc0-f339-48bb-a7ba-08fc0f4c4317 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:24.162686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c1cc0-f339-48bb-a7ba-08fc0f4c4317 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:24.171392 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 214/446] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:54:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:24.172208 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c1cc0-f339-48bb-a7ba-08fc0f4c4317 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:24.226456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 233/447] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:54:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:54:24.227341 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadb3af-6fa6-47a2-87e5-93e541ff0ed2 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:24.277846 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c1cc0-f339-48bb-a7ba-08fc0f4c4317 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:24.311722 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10239f21-4c3d-4b7a-9197-df656c245846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:24.312338 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10239f21-4c3d-4b7a-9197-df656c245846 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:24.312626 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10239f21-4c3d-4b7a-9197-df656c245846 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:24.312936 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10239f21-4c3d-4b7a-9197-df656c245846 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:24.313720 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10239f21-4c3d-4b7a-9197-df656c245846 None admin] RBAC: Adding query filter params (group.id=131b1e47e2d04feb9e09af36fbbd11ac, role.id=b0e0084bb31348a582616b1421f3b10c, scope.project.id=1952481417bf4a2cad0b08d5a7676b61) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:24.314039 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10239f21-4c3d-4b7a-9197-df656c245846 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:24.315580 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10239f21-4c3d-4b7a-9197-df656c245846 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:24.320462 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:24.321564 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 215/448] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Aug 3 04:54:24 2022] GET /identity/v3/role_assignments?group.id=131b1e47e2d04feb9e09af36fbbd11ac&scope.project.id=1952481417bf4a2cad0b08d5a7676b61&role.id=b0e0084bb31348a582616b1421f3b10c => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:24.322348 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10239f21-4c3d-4b7a-9197-df656c245846 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:24.323608 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadb3af-6fa6-47a2-87e5-93e541ff0ed2 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:24.337322 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:24.366177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05197116-6262-4707-9fc5-ccafae2ca5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:24.367232 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05197116-6262-4707-9fc5-ccafae2ca5fa None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:24.367638 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05197116-6262-4707-9fc5-ccafae2ca5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:24.367961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05197116-6262-4707-9fc5-ccafae2ca5fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:24.441949 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10239f21-4c3d-4b7a-9197-df656c245846 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:24.443653 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05197116-6262-4707-9fc5-ccafae2ca5fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:24.445148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05197116-6262-4707-9fc5-ccafae2ca5fa None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:24.481271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588abeb-f495-4519-903a-f8c40e3af133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:24.482387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588abeb-f495-4519-903a-f8c40e3af133 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:24.482387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588abeb-f495-4519-903a-f8c40e3af133 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:24.482552 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588abeb-f495-4519-903a-f8c40e3af133 None admin] PATH_INFO: `/v3/projects/255c31b4828b499abeb900572c1ce254` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:24.483611 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e588abeb-f495-4519-903a-f8c40e3af133 None admin] RBAC: Authorizing `identity:get_project(project_id=255c31b4828b499abeb900572c1ce254)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:24.485576 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e588abeb-f495-4519-903a-f8c40e3af133 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:24.487316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 216/449] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:54:24 2022] GET /identity/v3/projects/255c31b4828b499abeb900572c1ce254 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:24.492836 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588abeb-f495-4519-903a-f8c40e3af133 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:24.565525 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:24.570189 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 234/450] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:54:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:54:24.575907 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05197116-6262-4707-9fc5-ccafae2ca5fa None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:24.585794 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:24.615422 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588abeb-f495-4519-903a-f8c40e3af133 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:24.658257 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad1480-e6fe-4fc2-b113-59bdae46b476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:24.659216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad1480-e6fe-4fc2-b113-59bdae46b476 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:24.659216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad1480-e6fe-4fc2-b113-59bdae46b476 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:24.659492 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad1480-e6fe-4fc2-b113-59bdae46b476 None admin] PATH_INFO: `/v3/groups/131b1e47e2d04feb9e09af36fbbd11ac` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:24.660685 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad1480-e6fe-4fc2-b113-59bdae46b476 None admin] RBAC: Authorizing `identity:get_group(group_id=131b1e47e2d04feb9e09af36fbbd11ac)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:24.662136 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad1480-e6fe-4fc2-b113-59bdae46b476 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:24.664543 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 217/451] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Aug 3 04:54:24 2022] GET /identity/v3/groups/131b1e47e2d04feb9e09af36fbbd11ac => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:24.670869 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad1480-e6fe-4fc2-b113-59bdae46b476 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:24.704439 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05197116-6262-4707-9fc5-ccafae2ca5fa None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:24.743343 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89882673-ce29-49cb-a1cd-98135c4d27d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:24.743992 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89882673-ce29-49cb-a1cd-98135c4d27d4 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:24.744292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89882673-ce29-49cb-a1cd-98135c4d27d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:24.744601 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89882673-ce29-49cb-a1cd-98135c4d27d4 None admin] PATH_INFO: `/v3/projects/6917683e025f412781fd852422d27ce7` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:24.745867 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89882673-ce29-49cb-a1cd-98135c4d27d4 None admin] RBAC: Authorizing `identity:get_project(project_id=6917683e025f412781fd852422d27ce7)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:24.747463 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89882673-ce29-49cb-a1cd-98135c4d27d4 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:24.749132 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 235/452] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:54:24 2022] GET /identity/v3/projects/6917683e025f412781fd852422d27ce7 => generated 344 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:24.754982 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89882673-ce29-49cb-a1cd-98135c4d27d4 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:24.792332 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad1480-e6fe-4fc2-b113-59bdae46b476 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:24.835351 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30496e29-0f04-4a0b-9eb0-6c00ece19391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:24.835981 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30496e29-0f04-4a0b-9eb0-6c00ece19391 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:24.836270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30496e29-0f04-4a0b-9eb0-6c00ece19391 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:24.837436 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30496e29-0f04-4a0b-9eb0-6c00ece19391 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:24.837560 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30496e29-0f04-4a0b-9eb0-6c00ece19391 None admin] RBAC: Adding query filter params (group.id=131b1e47e2d04feb9e09af36fbbd11ac, role.id=561445e71aff40a0af7b01c539d3cfd9, scope.project.id=255c31b4828b499abeb900572c1ce254) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:24.837797 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30496e29-0f04-4a0b-9eb0-6c00ece19391 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:24.839297 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30496e29-0f04-4a0b-9eb0-6c00ece19391 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:24.840799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:24.846927 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 218/453] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Aug 3 04:54:24 2022] GET /identity/v3/role_assignments?group.id=131b1e47e2d04feb9e09af36fbbd11ac&scope.project.id=255c31b4828b499abeb900572c1ce254&role.id=561445e71aff40a0af7b01c539d3cfd9 => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:24.892323 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89882673-ce29-49cb-a1cd-98135c4d27d4 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:24.934157 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf73d7-44bb-4d64-b2e3-700845c81b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:24.934823 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf73d7-44bb-4d64-b2e3-700845c81b2c None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:24.935141 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf73d7-44bb-4d64-b2e3-700845c81b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:24.935415 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf73d7-44bb-4d64-b2e3-700845c81b2c None admin] PATH_INFO: `/v3/groups/90fa737b91fd4eaabcfc7a1814d2221c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:24.936659 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf73d7-44bb-4d64-b2e3-700845c81b2c None admin] RBAC: Authorizing `identity:get_group(group_id=90fa737b91fd4eaabcfc7a1814d2221c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:24.938177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf73d7-44bb-4d64-b2e3-700845c81b2c None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:24.940026 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 236/454] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Aug 3 04:54:24 2022] GET /identity/v3/groups/90fa737b91fd4eaabcfc7a1814d2221c => generated 229 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:24.946098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30496e29-0f04-4a0b-9eb0-6c00ece19391 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:24.961116 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:25.060871 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30496e29-0f04-4a0b-9eb0-6c00ece19391 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:25.096206 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda6602-d231-44fc-a609-e8845249dbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:25.096825 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda6602-d231-44fc-a609-e8845249dbd5 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:25.097101 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda6602-d231-44fc-a609-e8845249dbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:25.097373 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda6602-d231-44fc-a609-e8845249dbd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:25.098162 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcda6602-d231-44fc-a609-e8845249dbd5 None admin] RBAC: Adding query filter params (group.id=90fa737b91fd4eaabcfc7a1814d2221c, role.id=e39d4ecea4c64d5794e1079c9be3ad36, scope.project.id=6917683e025f412781fd852422d27ce7) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:25.098465 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcda6602-d231-44fc-a609-e8845249dbd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:25.099945 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcda6602-d231-44fc-a609-e8845249dbd5 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:25.105928 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 219/455] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Aug 3 04:54:24 2022] GET /identity/v3/role_assignments?group.id=90fa737b91fd4eaabcfc7a1814d2221c&scope.project.id=6917683e025f412781fd852422d27ce7&role.id=e39d4ecea4c64d5794e1079c9be3ad36 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:27.666664 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe584e43-659b-4a8c-94d4-ad33c6641b92 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:27.667570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe584e43-659b-4a8c-94d4-ad33c6641b92 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:27.668188 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe584e43-659b-4a8c-94d4-ad33c6641b92 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:27.669950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 237/456] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:27.675760 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd492a-d011-49ae-88a0-2a3917b20fb0 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:27.676580 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd492a-d011-49ae-88a0-2a3917b20fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:27.677111 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd492a-d011-49ae-88a0-2a3917b20fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:27.704630 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-dbbd492a-d011-49ae-88a0-2a3917b20fb0 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:27.706128 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-dbbd492a-d011-49ae-88a0-2a3917b20fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:27.771152 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31cb73c-987a-4d60-96f2-57472a9cb68c None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:27.771803 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31cb73c-987a-4d60-96f2-57472a9cb68c None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:27.772284 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31cb73c-987a-4d60-96f2-57472a9cb68c None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:27.774090 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 238/457] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:27.779872 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45f3fc-1884-494b-ba37-a3d12cc8e1f1 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:27.780417 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45f3fc-1884-494b-ba37-a3d12cc8e1f1 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:27.780900 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45f3fc-1884-494b-ba37-a3d12cc8e1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:27.801065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-af45f3fc-1884-494b-ba37-a3d12cc8e1f1 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:27.802581 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-af45f3fc-1884-494b-ba37-a3d12cc8e1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:27.807299 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 220/458] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:27.852142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbd492a-d011-49ae-88a0-2a3917b20fb0 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:27.860541 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:27.876968 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 239/459] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 04:54:27.901532 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45f3fc-1884-494b-ba37-a3d12cc8e1f1 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:27.972033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:28.010256 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45f3fc-1884-494b-ba37-a3d12cc8e1f1 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:28.050173 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb5b66-e3bf-4a26-ace6-7e35dd4027b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:28.051237 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb5b66-e3bf-4a26-ace6-7e35dd4027b7 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:28.051743 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb5b66-e3bf-4a26-ace6-7e35dd4027b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:28.052368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb5b66-e3bf-4a26-ace6-7e35dd4027b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:28.064321 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb5b66-e3bf-4a26-ace6-7e35dd4027b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:28.066514 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb5b66-e3bf-4a26-ace6-7e35dd4027b7 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:28.080728 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbd492a-d011-49ae-88a0-2a3917b20fb0 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:28.083878 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-6dcb5b66-e3bf-4a26-ace6-7e35dd4027b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:28.083878 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:28.083878 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:28.083878 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:28.083878 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:28.083878 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:28.083878 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:28.083878 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:28.083878 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:28.083878 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:28.083878 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:28.083878 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:28.083878 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:28.083878 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:28.083878 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:28.084741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:28.084741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:28.084741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:28.084741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:28.084741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:28.084741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:28.084741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:28.084741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:28.084741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:28.084741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:28.084741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:28.084741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:28.084741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:28.084741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:28.084741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:28.084741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:28.084741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:28.085661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:28.085661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:28.085661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:28.085661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:28.088905 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 240/460] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:28.095652 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb5b66-e3bf-4a26-ace6-7e35dd4027b7 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:28.133403 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae71799-3321-4de2-b4ff-9b81a1000760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:28.134354 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae71799-3321-4de2-b4ff-9b81a1000760 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:28.134980 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae71799-3321-4de2-b4ff-9b81a1000760 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:28.135418 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae71799-3321-4de2-b4ff-9b81a1000760 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:28.147248 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae71799-3321-4de2-b4ff-9b81a1000760 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:28.149024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae71799-3321-4de2-b4ff-9b81a1000760 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:28.166673 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-eae71799-3321-4de2-b4ff-9b81a1000760 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:28.166673 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:28.166673 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:28.166673 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:28.166673 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:28.166673 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:28.166673 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:28.166673 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:28.166673 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:28.166673 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:28.166673 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:28.166673 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:28.166673 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:28.166673 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:28.166673 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:28.167744 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:28.167744 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:28.167744 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:28.167744 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:28.167744 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:28.167744 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:28.167744 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:28.167744 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:28.167744 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:28.167744 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:28.167744 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:28.167744 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:28.167744 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:28.167744 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:28.167744 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:28.167744 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:28.167744 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:28.168734 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:28.168734 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:28.168734 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:28.168734 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:28.170688 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 221/461] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:28.171915 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f421c-3a89-47fc-a28b-611f3d486771 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:28.172236 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f421c-3a89-47fc-a28b-611f3d486771 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:28.172509 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f421c-3a89-47fc-a28b-611f3d486771 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:28.173722 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 222/462] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:28.174725 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e8f7c-2811-4a4d-9737-9b14c59dd585 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:28.175031 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e8f7c-2811-4a4d-9737-9b14c59dd585 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:28.179260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e8f7c-2811-4a4d-9737-9b14c59dd585 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:28.180646 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 223/463] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:28.181534 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e8f7c-2811-4a4d-9737-9b14c59dd585 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:28.216418 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb5b66-e3bf-4a26-ace6-7e35dd4027b7 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:28.234091 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:28.243830 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:28.251711 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af956176-431c-49e4-976c-b064b6f066b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:28.252340 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af956176-431c-49e4-976c-b064b6f066b0 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:28.252626 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af956176-431c-49e4-976c-b064b6f066b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:28.252904 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af956176-431c-49e4-976c-b064b6f066b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:28.253595 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af956176-431c-49e4-976c-b064b6f066b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:28.253954 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af956176-431c-49e4-976c-b064b6f066b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:28.255391 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af956176-431c-49e4-976c-b064b6f066b0 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:28.268425 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 241/464] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:28.270534 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81085465-8958-473a-9ea4-3d02bedd3122 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:28.270865 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81085465-8958-473a-9ea4-3d02bedd3122 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:28.271150 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81085465-8958-473a-9ea4-3d02bedd3122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:28.289554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-81085465-8958-473a-9ea4-3d02bedd3122 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:28.290763 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-81085465-8958-473a-9ea4-3d02bedd3122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:28.300769 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e8f7c-2811-4a4d-9737-9b14c59dd585 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:28.337592 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7437995-a809-43d7-ad19-bdadf2ccb14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:28.338221 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7437995-a809-43d7-ad19-bdadf2ccb14e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:28.338507 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7437995-a809-43d7-ad19-bdadf2ccb14e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:28.338783 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7437995-a809-43d7-ad19-bdadf2ccb14e None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:28.339508 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7437995-a809-43d7-ad19-bdadf2ccb14e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:28.339816 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7437995-a809-43d7-ad19-bdadf2ccb14e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:28.341189 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7437995-a809-43d7-ad19-bdadf2ccb14e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:28.358437 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 224/465] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:28.359876 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd8350-9497-42eb-8215-e62aebee2876 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:28.360169 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd8350-9497-42eb-8215-e62aebee2876 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:28.371093 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd8350-9497-42eb-8215-e62aebee2876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:28.371227 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 242/466] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:28.371227 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81085465-8958-473a-9ea4-3d02bedd3122 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:28.384118 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-e7dd8350-9497-42eb-8215-e62aebee2876 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:28.385330 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e7dd8350-9497-42eb-8215-e62aebee2876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:28.471010 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 225/467] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:28.472250 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dd8350-9497-42eb-8215-e62aebee2876 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:28.483866 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81085465-8958-473a-9ea4-3d02bedd3122 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:28.495080 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:28.521376 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddde381-a298-464c-bfdf-668ab06d2e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:28.521822 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:28.522679 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddde381-a298-464c-bfdf-668ab06d2e7d None admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:28.523070 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddde381-a298-464c-bfdf-668ab06d2e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:28.523330 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddde381-a298-464c-bfdf-668ab06d2e7d None admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:28.523940 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddde381-a298-464c-bfdf-668ab06d2e7d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:28.525696 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddde381-a298-464c-bfdf-668ab06d2e7d None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:28.578797 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dd8350-9497-42eb-8215-e62aebee2876 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:28.615145 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 243/468] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 3 04:54:28 2022] POST /identity/v3/users => generated 248 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:54:28.615145 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddde381-a298-464c-bfdf-668ab06d2e7d None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:28.621909 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c3e52-ed52-4b5c-a1f4-f3ce89c3af62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:28.622560 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c3e52-ed52-4b5c-a1f4-f3ce89c3af62 None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:28.622865 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c3e52-ed52-4b5c-a1f4-f3ce89c3af62 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:28.623184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c3e52-ed52-4b5c-a1f4-f3ce89c3af62 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:28.623886 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c3e52-ed52-4b5c-a1f4-f3ce89c3af62 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:28.626041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c3e52-ed52-4b5c-a1f4-f3ce89c3af62 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:28.718280 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 226/469] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 3 04:54:28 2022] POST /identity/v3/users => generated 246 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:54:28.719518 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c3e52-ed52-4b5c-a1f4-f3ce89c3af62 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:28.759424 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddde381-a298-464c-bfdf-668ab06d2e7d None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:28.807472 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f8d2c-e0bd-4d2d-ad4e-516afbb2f4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:28.809202 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f8d2c-e0bd-4d2d-ad4e-516afbb2f4f6 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:28.809679 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f8d2c-e0bd-4d2d-ad4e-516afbb2f4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:28.810214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f8d2c-e0bd-4d2d-ad4e-516afbb2f4f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:28.826745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730f8d2c-e0bd-4d2d-ad4e-516afbb2f4f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:28.828654 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730f8d2c-e0bd-4d2d-ad4e-516afbb2f4f6 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:28.845503 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-730f8d2c-e0bd-4d2d-ad4e-516afbb2f4f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:28.845503 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:28.845503 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:28.845503 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:28.845503 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:28.845503 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:28.845503 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:28.845503 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:28.845503 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:28.845503 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:28.845503 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:28.845503 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:28.845503 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:28.845503 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:28.845503 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:28.846411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:28.846411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:28.846411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:28.846411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:28.846411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:28.846411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:28.846411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:28.846411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:28.846411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:28.846411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:28.846411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:28.846411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:28.846411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:28.846411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:28.846411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:28.846411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:28.846411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:28.847306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:28.847306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:28.847306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:28.847306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:28.849712 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 244/470] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:28.855267 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f8d2c-e0bd-4d2d-ad4e-516afbb2f4f6 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:28.878780 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:28.883482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c3e52-ed52-4b5c-a1f4-f3ce89c3af62 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:28.885520 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:28.931772 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14877ba7-1a66-4320-b309-08ed810cebb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:28.932783 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14877ba7-1a66-4320-b309-08ed810cebb7 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:28.933654 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14877ba7-1a66-4320-b309-08ed810cebb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:28.934238 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14877ba7-1a66-4320-b309-08ed810cebb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:28.951107 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14877ba7-1a66-4320-b309-08ed810cebb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:28.952941 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14877ba7-1a66-4320-b309-08ed810cebb7 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:28.971124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-14877ba7-1a66-4320-b309-08ed810cebb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:28.971124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:28.971124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:28.971124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:28.971124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:28.971124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:28.971124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:28.971124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:28.971124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:28.971124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:28.971124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:28.971124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:28.971124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:28.971124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:28.971124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:28.972066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:28.972066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:28.972066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:28.972066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:28.972066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:28.972066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:28.972066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:28.972066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:28.972066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:28.972066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:28.972066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:28.972066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:28.972066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:28.972066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:28.972066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:28.972066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:28.972066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:28.972903 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:28.972903 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:28.972903 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:28.972903 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:28.974927 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 227/471] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:28.981156 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14877ba7-1a66-4320-b309-08ed810cebb7 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:29.003772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f8d2c-e0bd-4d2d-ad4e-516afbb2f4f6 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:29.045737 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cdc897-32a6-4c6a-a451-f510aa2e1270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:29.046654 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cdc897-32a6-4c6a-a451-f510aa2e1270 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:29.047675 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cdc897-32a6-4c6a-a451-f510aa2e1270 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:29.048100 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cdc897-32a6-4c6a-a451-f510aa2e1270 None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:29.048943 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cdc897-32a6-4c6a-a451-f510aa2e1270 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:29.049475 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cdc897-32a6-4c6a-a451-f510aa2e1270 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:29.051273 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cdc897-32a6-4c6a-a451-f510aa2e1270 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:29.066262 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 245/472] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:29.071729 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80010a7e-3eeb-47b1-9e73-12f4974cd675 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:29.072050 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80010a7e-3eeb-47b1-9e73-12f4974cd675 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:29.072300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80010a7e-3eeb-47b1-9e73-12f4974cd675 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:29.073895 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 246/473] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:29.075709 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80010a7e-3eeb-47b1-9e73-12f4974cd675 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:29.105635 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14877ba7-1a66-4320-b309-08ed810cebb7 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:29.147680 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd67b6-406d-49d3-ad80-c06d325af304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:29.148928 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd67b6-406d-49d3-ad80-c06d325af304 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:29.149821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd67b6-406d-49d3-ad80-c06d325af304 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:29.150324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd67b6-406d-49d3-ad80-c06d325af304 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:29.151707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd67b6-406d-49d3-ad80-c06d325af304 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:29.152334 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd67b6-406d-49d3-ad80-c06d325af304 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:29.154448 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd67b6-406d-49d3-ad80-c06d325af304 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:29.171138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 228/474] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:29.173299 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad3e95-de2a-4f5c-b07a-5157481b7237 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:29.173799 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad3e95-de2a-4f5c-b07a-5157481b7237 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:29.174273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad3e95-de2a-4f5c-b07a-5157481b7237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:29.196247 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-c0ad3e95-de2a-4f5c-b07a-5157481b7237 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:29.198354 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c0ad3e95-de2a-4f5c-b07a-5157481b7237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:29.207232 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:29.210180 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:29.213993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80010a7e-3eeb-47b1-9e73-12f4974cd675 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:29.258325 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76806e2-3553-4cd8-9694-d667671fddb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:29.259080 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76806e2-3553-4cd8-9694-d667671fddb1 None admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:29.259263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76806e2-3553-4cd8-9694-d667671fddb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:29.259533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76806e2-3553-4cd8-9694-d667671fddb1 None admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:29.260605 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76806e2-3553-4cd8-9694-d667671fddb1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:29.262361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76806e2-3553-4cd8-9694-d667671fddb1 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:29.283983 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 229/475] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:29.285108 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad3e95-de2a-4f5c-b07a-5157481b7237 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:29.361940 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 247/476] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 3 04:54:29 2022] POST /identity/v3/users => generated 248 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 04:54:29.364427 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76806e2-3553-4cd8-9694-d667671fddb1 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:29.439764 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad3e95-de2a-4f5c-b07a-5157481b7237 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:29.507417 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b74bd1-26f4-46b7-86e7-39132a0ebe69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:29.508264 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b74bd1-26f4-46b7-86e7-39132a0ebe69 None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:29.508393 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b74bd1-26f4-46b7-86e7-39132a0ebe69 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:29.508581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b74bd1-26f4-46b7-86e7-39132a0ebe69 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:29.509283 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b74bd1-26f4-46b7-86e7-39132a0ebe69 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:29.510762 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b74bd1-26f4-46b7-86e7-39132a0ebe69 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:29.512140 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76806e2-3553-4cd8-9694-d667671fddb1 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:29.550727 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:29.560173 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159f012-57b5-420e-ab8f-b6c9c99b78b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:29.560173 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159f012-57b5-420e-ab8f-b6c9c99b78b9 None admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:29.560173 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159f012-57b5-420e-ab8f-b6c9c99b78b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:29.560478 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159f012-57b5-420e-ab8f-b6c9c99b78b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:29.561453 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9159f012-57b5-420e-ab8f-b6c9c99b78b9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:29.562543 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9159f012-57b5-420e-ab8f-b6c9c99b78b9 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:29.603505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.sql.core [None req-9159f012-57b5-420e-ab8f-b6c9c99b78b9 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 03 04:54:29.603505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 03 04:54:29.603505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [parameters: {'id': '03d9074c98e34531902e186d290a15f2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 03 04:54:29.603505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80356) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 03 04:54:29.620997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-9159f012-57b5-420e-ab8f-b6c9c99b78b9 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 03 04:54:29.620997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:29.620997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 03 04:54:29.620997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 03 04:54:29.620997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 03 04:54:29.620997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 03 04:54:29.620997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 03 04:54:29.620997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application result = self._query(query) Aug 03 04:54:29.620997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 03 04:54:29.620997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application conn.query(q) Aug 03 04:54:29.620997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 03 04:54:29.620997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 03 04:54:29.620997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 03 04:54:29.620997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application result.read() Aug 03 04:54:29.620997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 03 04:54:29.621971 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 03 04:54:29.621971 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 03 04:54:29.621971 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 03 04:54:29.621971 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 03 04:54:29.621971 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 03 04:54:29.621971 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 03 04:54:29.621971 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 03 04:54:29.621971 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 03 04:54:29.621971 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:29.621971 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 03 04:54:29.621971 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:29.621971 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:29.621971 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 03 04:54:29.621971 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 03 04:54:29.621971 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 03 04:54:29.621971 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return ref.to_dict() Aug 03 04:54:29.621971 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 03 04:54:29.621971 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application next(self.gen) Aug 03 04:54:29.622918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 03 04:54:29.622918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application yield resource Aug 03 04:54:29.622918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 03 04:54:29.622918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application next(self.gen) Aug 03 04:54:29.622918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 03 04:54:29.622918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self.session.rollback() Aug 03 04:54:29.622918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 03 04:54:29.622918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self.force_reraise() Aug 03 04:54:29.622918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 03 04:54:29.622918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise self.value Aug 03 04:54:29.622918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 03 04:54:29.622918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 03 04:54:29.622918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 03 04:54:29.622918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application session.commit() Aug 03 04:54:29.622918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 03 04:54:29.622918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 03 04:54:29.622918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 03 04:54:29.623803 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self._prepare_impl() Aug 03 04:54:29.623803 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 03 04:54:29.623803 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self.session.flush() Aug 03 04:54:29.623803 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 03 04:54:29.623803 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self._flush(objects) Aug 03 04:54:29.623803 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 03 04:54:29.623803 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 03 04:54:29.623803 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 03 04:54:29.623803 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application compat.raise_( Aug 03 04:54:29.623803 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 03 04:54:29.623803 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception Aug 03 04:54:29.623803 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 03 04:54:29.623803 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application flush_context.execute() Aug 03 04:54:29.623803 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 03 04:54:29.623803 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rec.execute(self) Aug 03 04:54:29.623803 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 03 04:54:29.623803 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 03 04:54:29.624698 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 03 04:54:29.624698 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 03 04:54:29.624698 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 03 04:54:29.624698 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 03 04:54:29.624698 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 03 04:54:29.624698 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 03 04:54:29.624698 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 03 04:54:29.624698 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 03 04:54:29.624698 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 03 04:54:29.624698 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 03 04:54:29.624698 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 03 04:54:29.624698 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 03 04:54:29.624698 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 03 04:54:29.624698 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 03 04:54:29.624698 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 03 04:54:29.624698 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception Aug 03 04:54:29.625590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 03 04:54:29.625590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 03 04:54:29.625590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 03 04:54:29.625590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 03 04:54:29.625590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 03 04:54:29.625590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application result = self._query(query) Aug 03 04:54:29.625590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 03 04:54:29.625590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application conn.query(q) Aug 03 04:54:29.625590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 03 04:54:29.625590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 03 04:54:29.625590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 03 04:54:29.625590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application result.read() Aug 03 04:54:29.625590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 03 04:54:29.625590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 03 04:54:29.625590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 03 04:54:29.625590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 03 04:54:29.626432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 03 04:54:29.626432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 03 04:54:29.626432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 03 04:54:29.626432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 03 04:54:29.626432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 03 04:54:29.626432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 03 04:54:29.626432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [parameters: {'id': '03d9074c98e34531902e186d290a15f2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 03 04:54:29.626432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 03 04:54:29.626432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:29.626432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:54:29.626432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:29.626432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:29.626432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:29.626432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:29.626432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:29.626432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:29.632142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:29.632142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:29.632142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:29.632142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:29.632142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:29.632142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:29.632142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 03 04:54:29.632142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 03 04:54:29.632142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:29.632142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:29.632142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 03 04:54:29.632142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 03 04:54:29.632142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 03 04:54:29.632142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 03 04:54:29.632142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 03 04:54:29.632142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:29.633166 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 248/477] 10.222.0.40 () {60 vars in 1131 bytes} [Wed Aug 3 04:54:29 2022] POST /identity/v3/roles => generated 144 bytes in 258 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:29.633166 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159f012-57b5-420e-ab8f-b6c9c99b78b9 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:29.633288 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:29.648182 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 230/478] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 3 04:54:29 2022] POST /identity/v3/users => generated 249 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:54:29.759375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159f012-57b5-420e-ab8f-b6c9c99b78b9 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:29.799685 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3423d-8220-4e07-95d4-04c588b209f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:29.800533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3423d-8220-4e07-95d4-04c588b209f4 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:29.800652 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3423d-8220-4e07-95d4-04c588b209f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:29.800926 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3423d-8220-4e07-95d4-04c588b209f4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:29.813786 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3423d-8220-4e07-95d4-04c588b209f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:29.823768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-6cd3423d-8220-4e07-95d4-04c588b209f4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 04:54:29.823768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:29.823768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:29.823768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:29.823768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:29.823768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:29.823768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:29.823768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:29.823768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:29.823768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:29.823768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:29.823768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:29.823768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:29.823768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:29.823768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:29.823768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:29.824727 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:29.824727 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:29.824727 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:29.824727 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:29.824727 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:29.824727 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:29.824727 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:29.824727 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:29.824727 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:29.824727 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:29.824727 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:29.824727 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:29.824727 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:29.824727 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:29.824727 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:29.824727 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:29.825518 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:29.825518 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:29.825518 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:29.825518 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:29.825518 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:29.825518 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:29.825518 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:29.825518 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:29.825518 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:29.825518 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:29.825518 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 04:54:29.825518 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:29.825518 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 249/479] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 3 04:54:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:54:29.831728 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b74bd1-26f4-46b7-86e7-39132a0ebe69 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:29.952209 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b74bd1-26f4-46b7-86e7-39132a0ebe69 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:29.998065 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6bc390-6fd4-4f24-9453-dc3edbde5183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:29.998695 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6bc390-6fd4-4f24-9453-dc3edbde5183 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:29.999015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6bc390-6fd4-4f24-9453-dc3edbde5183 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:29.999284 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6bc390-6fd4-4f24-9453-dc3edbde5183 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:30.000542 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6bc390-6fd4-4f24-9453-dc3edbde5183 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:30.000856 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6bc390-6fd4-4f24-9453-dc3edbde5183 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:30.002262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6bc390-6fd4-4f24-9453-dc3edbde5183 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:30.026430 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 231/480] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:54:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:31.052790 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879bb27-6055-4505-8b39-d44be944c613 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:31.053313 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879bb27-6055-4505-8b39-d44be944c613 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:31.053313 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879bb27-6055-4505-8b39-d44be944c613 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:31.054685 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 250/481] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:31.059868 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb585891-3e91-46aa-976b-c0ff2ac139be None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:31.060289 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb585891-3e91-46aa-976b-c0ff2ac139be None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:31.060577 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb585891-3e91-46aa-976b-c0ff2ac139be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:31.088516 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-bb585891-3e91-46aa-976b-c0ff2ac139be None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:31.089715 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-bb585891-3e91-46aa-976b-c0ff2ac139be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:31.166272 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:31.180386 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 232/482] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 04:54:31.215662 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d879bb27-6055-4505-8b39-d44be944c613 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:31.257079 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba950078-73b4-44ec-88f4-0e4d0202901c None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:31.257420 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba950078-73b4-44ec-88f4-0e4d0202901c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:31.257630 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba950078-73b4-44ec-88f4-0e4d0202901c None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:31.259022 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 233/483] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:54:31.264435 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b585a-64ff-44ca-a84b-fa65aac3300e None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:31.264967 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b585a-64ff-44ca-a84b-fa65aac3300e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:31.265451 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b585a-64ff-44ca-a84b-fa65aac3300e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:31.294092 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-805b585a-64ff-44ca-a84b-fa65aac3300e None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:31.295823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-805b585a-64ff-44ca-a84b-fa65aac3300e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:31.312454 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:31.387565 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 234/484] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:31.414421 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b585a-64ff-44ca-a84b-fa65aac3300e None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:31.414738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d879bb27-6055-4505-8b39-d44be944c613 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:31.479731 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a34592-bd43-4cd6-a96d-61938ade9720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:31.480385 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a34592-bd43-4cd6-a96d-61938ade9720 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:31.480670 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a34592-bd43-4cd6-a96d-61938ade9720 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:31.480946 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a34592-bd43-4cd6-a96d-61938ade9720 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:31.501811 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a34592-bd43-4cd6-a96d-61938ade9720 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:31.506783 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-81a34592-bd43-4cd6-a96d-61938ade9720 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:31.506783 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:31.506783 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:31.506783 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:31.506783 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:31.506783 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:31.506783 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:31.506783 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:31.506783 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:31.506783 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:31.506783 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:31.506783 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:31.506783 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:31.506783 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:31.506783 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:31.506783 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:31.507691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:31.507691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:31.507691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:31.507691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:31.507691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:31.507691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:31.507691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:31.507691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:31.507691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:31.507691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:31.507691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:31.507691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:31.507691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:31.507691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:31.507691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:31.507691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:31.508490 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:31.508490 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:31.508490 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:31.508490 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:31.508490 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:31.508490 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:31.508490 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:31.508490 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:31.508490 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:31.508490 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:31.508490 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:31.508490 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:31.512022 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 251/485] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 3 04:54:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:31.518568 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a34592-bd43-4cd6-a96d-61938ade9720 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:31.583187 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b585a-64ff-44ca-a84b-fa65aac3300e None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:31.626306 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:31.653147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60bb2bb-e477-46df-a1d7-e2b18dca6d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:31.654305 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bb2bb-e477-46df-a1d7-e2b18dca6d2f None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:31.654653 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bb2bb-e477-46df-a1d7-e2b18dca6d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:31.655050 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bb2bb-e477-46df-a1d7-e2b18dca6d2f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:31.682022 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60bb2bb-e477-46df-a1d7-e2b18dca6d2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:31.687251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-b60bb2bb-e477-46df-a1d7-e2b18dca6d2f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:31.687251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:31.687251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:31.687251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:31.687251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:31.687251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:31.687251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:31.687251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:31.687251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:31.687251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:31.687251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:31.687251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:31.687251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:31.687251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:31.687251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:31.687251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:31.688245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:31.688245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:31.688245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:31.688245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:31.688245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:31.688245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:31.688245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:31.688245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:31.688245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:31.688245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:31.688245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:31.688245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:31.688245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:31.688245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:31.688245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:31.688245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:31.689045 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:31.689045 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:31.689045 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:31.689045 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:31.689045 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:31.689045 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:31.689045 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:31.689045 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:31.689045 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:31.689045 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:31.689045 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:31.689045 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:31.690245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 235/486] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 3 04:54:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:54:31.696259 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60bb2bb-e477-46df-a1d7-e2b18dca6d2f None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:31.735761 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a34592-bd43-4cd6-a96d-61938ade9720 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:31.772562 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:31.811358 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942b7e5-3e88-480d-b8da-78d73b9a4e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:31.812696 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942b7e5-3e88-480d-b8da-78d73b9a4e18 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:31.812990 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942b7e5-3e88-480d-b8da-78d73b9a4e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:31.813268 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942b7e5-3e88-480d-b8da-78d73b9a4e18 None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:31.813996 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a942b7e5-3e88-480d-b8da-78d73b9a4e18 None admin] RBAC: Adding query filter params (name=service) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:31.814304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a942b7e5-3e88-480d-b8da-78d73b9a4e18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:31.815956 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a942b7e5-3e88-480d-b8da-78d73b9a4e18 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:31.837039 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 252/487] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 3 04:54:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:31.842171 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942b7e5-3e88-480d-b8da-78d73b9a4e18 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:31.894816 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60bb2bb-e477-46df-a1d7-e2b18dca6d2f None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:31.951883 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a83eb-3525-4fdb-ad63-373d6310991d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:31.952389 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a83eb-3525-4fdb-ad63-373d6310991d None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:31.952679 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a83eb-3525-4fdb-ad63-373d6310991d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:31.953481 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a83eb-3525-4fdb-ad63-373d6310991d None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:31.955038 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5a83eb-3525-4fdb-ad63-373d6310991d None admin] RBAC: Adding query filter params (name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:31.956468 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5a83eb-3525-4fdb-ad63-373d6310991d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:31.958049 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5a83eb-3525-4fdb-ad63-373d6310991d None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:31.967799 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 236/488] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 3 04:54:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:31.972890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b77f3-1a2b-400c-96bb-8f98e44bd4d4 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:31.972890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b77f3-1a2b-400c-96bb-8f98e44bd4d4 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:31.972890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b77f3-1a2b-400c-96bb-8f98e44bd4d4 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:31.975450 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 237/489] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:31.976786 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76848e-32d2-4056-baca-bc6c71ed4f12 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:31.977071 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76848e-32d2-4056-baca-bc6c71ed4f12 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:31.977347 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76848e-32d2-4056-baca-bc6c71ed4f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:31.995707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-ed76848e-32d2-4056-baca-bc6c71ed4f12 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:31.996986 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ed76848e-32d2-4056-baca-bc6c71ed4f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:32.000154 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942b7e5-3e88-480d-b8da-78d73b9a4e18 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:32.038917 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b0454-885f-4377-8920-9044b5fc434a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:32.039914 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b0454-885f-4377-8920-9044b5fc434a None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:32.040420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b0454-885f-4377-8920-9044b5fc434a None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:32.040920 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b0454-885f-4377-8920-9044b5fc434a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:32.041406 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:32.054353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b0454-885f-4377-8920-9044b5fc434a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:32.056657 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b0454-885f-4377-8920-9044b5fc434a None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:32.072617 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-d26b0454-885f-4377-8920-9044b5fc434a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:32.072617 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:32.072617 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:32.072617 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:32.072617 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:32.072617 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:32.072617 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:32.072617 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:32.072617 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:32.072617 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:32.072617 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:32.072617 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:32.072617 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:32.072617 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:32.072617 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:32.073548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:32.073548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:32.073548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:32.073548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:32.073548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:32.073548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:32.073548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:32.073548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:32.073548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:32.073548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:32.073548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:32.073548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:32.073548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:32.073548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:32.073548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:32.073548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:32.073548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:32.074396 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:32.074396 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:32.074396 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:32.074396 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:32.075171 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 238/490] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:32.076064 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76848e-32d2-4056-baca-bc6c71ed4f12 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:32.077384 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 253/491] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:54:32.086126 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b0454-885f-4377-8920-9044b5fc434a None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:32.115785 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:32.198562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76848e-32d2-4056-baca-bc6c71ed4f12 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:32.201015 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b0454-885f-4377-8920-9044b5fc434a None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:32.233811 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4d106-b690-42d8-8f64-d5ac3c6c126d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:32.234497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4d106-b690-42d8-8f64-d5ac3c6c126d None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:32.234778 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4d106-b690-42d8-8f64-d5ac3c6c126d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:32.235178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4d106-b690-42d8-8f64-d5ac3c6c126d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:32.238429 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4f0d1-9b2b-4b6b-9d9a-fe98f7ba96a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:32.239083 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4f0d1-9b2b-4b6b-9d9a-fe98f7ba96a6 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:32.239366 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4f0d1-9b2b-4b6b-9d9a-fe98f7ba96a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:32.239645 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4f0d1-9b2b-4b6b-9d9a-fe98f7ba96a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:32.240339 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e4f0d1-9b2b-4b6b-9d9a-fe98f7ba96a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:32.240647 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e4f0d1-9b2b-4b6b-9d9a-fe98f7ba96a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:32.242049 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e4f0d1-9b2b-4b6b-9d9a-fe98f7ba96a6 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:32.246367 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4d106-b690-42d8-8f64-d5ac3c6c126d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:32.247929 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4d106-b690-42d8-8f64-d5ac3c6c126d None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:32.260966 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 254/492] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:32.261882 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4f0d1-9b2b-4b6b-9d9a-fe98f7ba96a6 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:32.264381 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-f8c4d106-b690-42d8-8f64-d5ac3c6c126d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:32.264381 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:32.264381 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:32.264381 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:32.264381 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:32.264381 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:32.264381 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:32.264381 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:32.264381 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:32.264381 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:32.264381 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:32.264381 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:32.264381 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:32.264381 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:32.264381 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:32.265227 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:32.265227 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:32.265227 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:32.265227 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:32.265227 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:32.265227 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:32.265227 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:32.265227 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:32.265227 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:32.265227 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:32.265227 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:32.265227 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:32.265227 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:32.265227 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:32.265227 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:32.265227 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:32.265227 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:32.266714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:32.266714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:32.266714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:32.266714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:32.266714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 239/493] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:32.267127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4d106-b690-42d8-8f64-d5ac3c6c126d None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:32.295638 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:32.379296 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4d106-b690-42d8-8f64-d5ac3c6c126d None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:32.379474 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:32.385347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4f0d1-9b2b-4b6b-9d9a-fe98f7ba96a6 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:32.415454 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f90a9-9330-47a4-a283-061ebe36bad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:32.416092 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f90a9-9330-47a4-a283-061ebe36bad0 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:32.416373 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f90a9-9330-47a4-a283-061ebe36bad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:32.416645 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f90a9-9330-47a4-a283-061ebe36bad0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:32.423303 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73282b8f-dc72-4470-aeb7-4b8819639e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:32.423950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73282b8f-dc72-4470-aeb7-4b8819639e7a None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:32.424234 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73282b8f-dc72-4470-aeb7-4b8819639e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:32.424514 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73282b8f-dc72-4470-aeb7-4b8819639e7a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:32.424750 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f90a9-9330-47a4-a283-061ebe36bad0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:32.426205 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f90a9-9330-47a4-a283-061ebe36bad0 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:32.435086 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73282b8f-dc72-4470-aeb7-4b8819639e7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:32.440084 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-a90f90a9-9330-47a4-a283-061ebe36bad0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 04:54:32.440084 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:32.440084 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:32.440084 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:32.440084 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:32.440084 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:32.440084 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:32.440084 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:32.440084 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:32.440084 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:32.440084 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:32.440084 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:32.440084 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:54:32.440084 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:54:32.440084 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:54:32.440084 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:54:32.440950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:32.440950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:32.440950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:54:32.440950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:32.440950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:54:32.440950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:32.440950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:32.440950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:32.440950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:32.440950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:32.440950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:32.440950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:32.440950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:32.440950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:32.440950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:32.440950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:32.440950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:32.440950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:32.442430 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:32.442430 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:32.442430 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:54:32.442430 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:54:32.442430 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:54:32.442430 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:54:32.442430 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:54:32.442430 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:54:32.442430 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 04:54:32.442430 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:32.443218 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-73282b8f-dc72-4470-aeb7-4b8819639e7a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:32.443218 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:32.443218 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:32.443218 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:32.443218 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:32.443218 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:32.443218 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:32.443218 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:32.443218 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:32.443218 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:32.443218 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:32.443218 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:32.443218 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:32.443218 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:32.443218 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:32.443218 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:32.444530 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:32.444530 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:32.444530 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:32.444530 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:32.444530 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:32.444530 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:32.444530 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:32.444530 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:32.444530 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:32.444530 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:32.444530 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:32.444530 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:32.444530 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:32.444530 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:32.444530 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:32.444530 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:32.447347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:32.447347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:32.447347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:32.447347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:32.447347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:32.447347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:32.447347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:32.447347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:32.447347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:32.447347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:32.447347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:32.447347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:32.447971 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 240/494] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:54:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:32.448074 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 255/495] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 3 04:54:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:32.448074 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7b6c2-818e-469d-88b7-708705172f70 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:32.448074 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7b6c2-818e-469d-88b7-708705172f70 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:32.448074 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7b6c2-818e-469d-88b7-708705172f70 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:32.448074 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 256/496] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:32.448074 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f7b6c2-818e-469d-88b7-708705172f70 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:32.448425 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f90a9-9330-47a4-a283-061ebe36bad0 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:32.536679 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:32.552580 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f90a9-9330-47a4-a283-061ebe36bad0 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:32.554384 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f7b6c2-818e-469d-88b7-708705172f70 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:32.588086 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffcc7ae-6a94-4187-9373-b56c4b41b93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:32.588704 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffcc7ae-6a94-4187-9373-b56c4b41b93c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:32.588984 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffcc7ae-6a94-4187-9373-b56c4b41b93c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:32.589144 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a366e1-374f-408f-8625-dcd262fabb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:32.589250 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffcc7ae-6a94-4187-9373-b56c4b41b93c None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:32.589763 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a366e1-374f-408f-8625-dcd262fabb5f None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:32.589926 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffcc7ae-6a94-4187-9373-b56c4b41b93c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:32.590072 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a366e1-374f-408f-8625-dcd262fabb5f None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:32.590224 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffcc7ae-6a94-4187-9373-b56c4b41b93c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:32.590321 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a366e1-374f-408f-8625-dcd262fabb5f None admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:32.591137 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a366e1-374f-408f-8625-dcd262fabb5f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:32.591510 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a366e1-374f-408f-8625-dcd262fabb5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:32.591662 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffcc7ae-6a94-4187-9373-b56c4b41b93c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:32.592965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a366e1-374f-408f-8625-dcd262fabb5f None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:32.607706 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 241/497] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:32.607706 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffcc7ae-6a94-4187-9373-b56c4b41b93c None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:32.635405 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 257/498] 10.222.0.40 () {56 vars in 1137 bytes} [Wed Aug 3 04:54:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:32.636586 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ade8c-b843-484f-84e5-639c7f607637 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:32.636877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ade8c-b843-484f-84e5-639c7f607637 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:32.637157 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ade8c-b843-484f-84e5-639c7f607637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:32.658821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-4d3ade8c-b843-484f-84e5-639c7f607637 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:32.660096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-4d3ade8c-b843-484f-84e5-639c7f607637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:32.662534 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:32.751000 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffcc7ae-6a94-4187-9373-b56c4b41b93c None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:32.752808 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 258/499] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:32.754028 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ade8c-b843-484f-84e5-639c7f607637 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:32.785957 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9111e-36f6-49c2-8c68-4912e0fde39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:32.786572 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9111e-36f6-49c2-8c68-4912e0fde39c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:32.786890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9111e-36f6-49c2-8c68-4912e0fde39c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:32.787171 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9111e-36f6-49c2-8c68-4912e0fde39c None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:32.787882 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f9111e-36f6-49c2-8c68-4912e0fde39c None admin] RBAC: Adding query filter params (name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:32.788187 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f9111e-36f6-49c2-8c68-4912e0fde39c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:32.789569 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f9111e-36f6-49c2-8c68-4912e0fde39c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:32.798357 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 242/500] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 3 04:54:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:32.798862 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9111e-36f6-49c2-8c68-4912e0fde39c None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:32.809348 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:32.860086 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ade8c-b843-484f-84e5-639c7f607637 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:32.893689 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ded5a6-fdd0-44fc-b7ca-fccfe1335c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:32.894271 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ded5a6-fdd0-44fc-b7ca-fccfe1335c69 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:32.894550 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ded5a6-fdd0-44fc-b7ca-fccfe1335c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:32.894823 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ded5a6-fdd0-44fc-b7ca-fccfe1335c69 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:32.898601 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9111e-36f6-49c2-8c68-4912e0fde39c None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:32.902205 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ded5a6-fdd0-44fc-b7ca-fccfe1335c69 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:32.903694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ded5a6-fdd0-44fc-b7ca-fccfe1335c69 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:32.905434 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:32.919418 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-64ded5a6-fdd0-44fc-b7ca-fccfe1335c69 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 04:54:32.919418 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:32.919418 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:32.919418 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:32.919418 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:32.919418 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:32.919418 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:32.919418 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:32.919418 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:32.919418 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:32.919418 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:32.919418 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:32.919418 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:54:32.919418 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:54:32.919418 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:54:32.919418 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:54:32.920380 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:32.920380 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:32.920380 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:54:32.920380 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:32.920380 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:54:32.920380 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:32.920380 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:32.920380 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:32.920380 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:32.920380 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:32.920380 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:32.920380 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:32.920380 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:32.920380 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:32.920380 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:32.920380 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:32.920380 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:32.920380 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:32.921278 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:32.921278 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:32.921278 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:54:32.921278 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:54:32.921278 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:54:32.921278 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:54:32.921278 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:54:32.921278 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:54:32.921278 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 04:54:32.921278 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:32.921278 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 259/501] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Aug 3 04:54:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:32.922912 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554a73e-fe7e-4672-be34-4f6ae0cfb616 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:32.922912 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554a73e-fe7e-4672-be34-4f6ae0cfb616 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:32.922912 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554a73e-fe7e-4672-be34-4f6ae0cfb616 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:32.924087 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 260/502] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:32.924768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554a73e-fe7e-4672-be34-4f6ae0cfb616 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:32.937897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4620a1f8-0e23-4a4f-906e-624b43ebde0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:32.938864 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4620a1f8-0e23-4a4f-906e-624b43ebde0e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:32.939151 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4620a1f8-0e23-4a4f-906e-624b43ebde0e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:32.939590 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4620a1f8-0e23-4a4f-906e-624b43ebde0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:33.009055 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4620a1f8-0e23-4a4f-906e-624b43ebde0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:33.010544 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4620a1f8-0e23-4a4f-906e-624b43ebde0e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:33.020746 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:33.029195 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554a73e-fe7e-4672-be34-4f6ae0cfb616 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:33.064057 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb68ef2-7eda-4ecd-a82e-27f034308c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:33.064701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb68ef2-7eda-4ecd-a82e-27f034308c1e None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:33.064969 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb68ef2-7eda-4ecd-a82e-27f034308c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:33.065253 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb68ef2-7eda-4ecd-a82e-27f034308c1e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:33.075538 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb68ef2-7eda-4ecd-a82e-27f034308c1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:33.080879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-4cb68ef2-7eda-4ecd-a82e-27f034308c1e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:33.080879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:33.080879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:33.080879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:33.080879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:33.080879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:33.080879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:33.080879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:33.080879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:33.080879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:33.080879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:33.080879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:33.080879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:33.080879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:33.080879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:33.080879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:33.081838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:33.081838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:33.081838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:33.081838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:33.081838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:33.081838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:33.081838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:33.081838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:33.081838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:33.081838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:33.081838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:33.081838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:33.081838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:33.081838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:33.081838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:33.081838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:33.082644 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:33.082644 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:33.082644 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:33.082644 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:33.082644 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:33.082644 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:33.082644 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:33.082644 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:33.082644 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:33.082644 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:33.082644 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:33.082644 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:33.082644 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 261/503] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 3 04:54:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:33.083635 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb68ef2-7eda-4ecd-a82e-27f034308c1e None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:33.110564 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 243/504] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:54:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:54:33.111437 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4620a1f8-0e23-4a4f-906e-624b43ebde0e None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:33.145360 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:33.187576 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb68ef2-7eda-4ecd-a82e-27f034308c1e None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:33.211289 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4620a1f8-0e23-4a4f-906e-624b43ebde0e None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:33.221088 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:33.222607 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291adec-7c8a-47ea-8ba7-1a0f4a10ec90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:33.223284 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291adec-7c8a-47ea-8ba7-1a0f4a10ec90 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:33.223574 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291adec-7c8a-47ea-8ba7-1a0f4a10ec90 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:33.223855 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291adec-7c8a-47ea-8ba7-1a0f4a10ec90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:33.234392 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291adec-7c8a-47ea-8ba7-1a0f4a10ec90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:33.235892 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291adec-7c8a-47ea-8ba7-1a0f4a10ec90 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:33.245960 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583c2fb-3bc0-492e-b7ba-ac84ca29c563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:33.246570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583c2fb-3bc0-492e-b7ba-ac84ca29c563 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:33.246876 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583c2fb-3bc0-492e-b7ba-ac84ca29c563 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:33.247160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583c2fb-3bc0-492e-b7ba-ac84ca29c563 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:33.247970 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c583c2fb-3bc0-492e-b7ba-ac84ca29c563 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:33.248281 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c583c2fb-3bc0-492e-b7ba-ac84ca29c563 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:33.249707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c583c2fb-3bc0-492e-b7ba-ac84ca29c563 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:33.250356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-4291adec-7c8a-47ea-8ba7-1a0f4a10ec90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:33.250356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:33.250356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:33.250356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:33.250356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:33.250356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:33.250356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:33.250356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:33.250356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:33.250356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:33.250356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:33.250356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:33.250356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:33.250356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:33.250356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:33.251346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:33.251346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:33.251346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:33.251346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:33.251346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:33.251346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:33.251346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:33.251346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:33.251346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:33.251346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:33.251346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:33.251346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:33.251346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:33.251346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:33.251346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:33.251346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:33.251346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:33.252230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:33.252230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:33.252230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:33.252230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:33.252230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 262/505] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:33.252690 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d05185-d3d3-40e4-9850-ea2b9c755a1d None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:33.252977 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d05185-d3d3-40e4-9850-ea2b9c755a1d None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:33.253256 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d05185-d3d3-40e4-9850-ea2b9c755a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:33.271200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-58d05185-d3d3-40e4-9850-ea2b9c755a1d None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:33.272455 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-58d05185-d3d3-40e4-9850-ea2b9c755a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:33.288782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 244/506] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Aug 3 04:54:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:33.289614 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583c2fb-3bc0-492e-b7ba-ac84ca29c563 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:33.346429 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 263/507] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:33.347311 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d05185-d3d3-40e4-9850-ea2b9c755a1d None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:33.389562 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:33.389982 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583c2fb-3bc0-492e-b7ba-ac84ca29c563 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:33.424705 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285759d0-0352-4908-b363-cadd21e7c02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:33.425246 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285759d0-0352-4908-b363-cadd21e7c02a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:33.425529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285759d0-0352-4908-b363-cadd21e7c02a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:33.425805 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285759d0-0352-4908-b363-cadd21e7c02a None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:33.426522 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285759d0-0352-4908-b363-cadd21e7c02a None admin] RBAC: Adding query filter params (name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:33.426823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285759d0-0352-4908-b363-cadd21e7c02a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:33.428245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285759d0-0352-4908-b363-cadd21e7c02a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:33.436108 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 245/508] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 3 04:54:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:33.436895 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285759d0-0352-4908-b363-cadd21e7c02a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:33.448044 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d05185-d3d3-40e4-9850-ea2b9c755a1d None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:33.467463 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:33.483497 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64f8d1-0678-4d4f-8c38-2df76d810cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:33.484163 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64f8d1-0678-4d4f-8c38-2df76d810cdc None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:33.484448 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64f8d1-0678-4d4f-8c38-2df76d810cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:33.484725 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64f8d1-0678-4d4f-8c38-2df76d810cdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:33.495124 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f64f8d1-0678-4d4f-8c38-2df76d810cdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:33.496592 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f64f8d1-0678-4d4f-8c38-2df76d810cdc None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:33.511694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-6f64f8d1-0678-4d4f-8c38-2df76d810cdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:33.511694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:33.511694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:33.511694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:33.511694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:33.511694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:33.511694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:33.511694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:33.511694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:33.511694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:33.511694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:33.511694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:33.511694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:33.511694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:33.511694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:33.512625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:33.512625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:33.512625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:33.512625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:33.512625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:33.512625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:33.512625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:33.512625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:33.512625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:33.512625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:33.512625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:33.512625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:33.512625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:33.512625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:33.512625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:33.512625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:33.512625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:33.513477 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:33.513477 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:33.513477 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:33.513477 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:33.513477 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 264/509] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:33.513745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64f8d1-0678-4d4f-8c38-2df76d810cdc None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:33.540617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285759d0-0352-4908-b363-cadd21e7c02a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:33.575366 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83dcf61-ec0c-44e3-a388-fc4da0f3ac53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:33.575971 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83dcf61-ec0c-44e3-a388-fc4da0f3ac53 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:33.576250 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83dcf61-ec0c-44e3-a388-fc4da0f3ac53 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:33.576524 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83dcf61-ec0c-44e3-a388-fc4da0f3ac53 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:33.577208 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83dcf61-ec0c-44e3-a388-fc4da0f3ac53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:33.577512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83dcf61-ec0c-44e3-a388-fc4da0f3ac53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:33.579549 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83dcf61-ec0c-44e3-a388-fc4da0f3ac53 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:33.591115 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 246/510] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:33.591900 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83dcf61-ec0c-44e3-a388-fc4da0f3ac53 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:33.607649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:33.616338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64f8d1-0678-4d4f-8c38-2df76d810cdc None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:33.652940 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d425354a-616f-41d2-9564-890760208c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:33.653564 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d425354a-616f-41d2-9564-890760208c4e None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:33.653849 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d425354a-616f-41d2-9564-890760208c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:33.654127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d425354a-616f-41d2-9564-890760208c4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:33.686044 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:33.698586 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83dcf61-ec0c-44e3-a388-fc4da0f3ac53 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:33.723804 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d425354a-616f-41d2-9564-890760208c4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:33.725262 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d425354a-616f-41d2-9564-890760208c4e None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:33.733769 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef652cc-872c-456c-850b-585a72692412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:33.734477 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef652cc-872c-456c-850b-585a72692412 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:33.734760 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef652cc-872c-456c-850b-585a72692412 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:33.735063 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef652cc-872c-456c-850b-585a72692412 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:33.745025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef652cc-872c-456c-850b-585a72692412 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:33.746472 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef652cc-872c-456c-850b-585a72692412 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:33.761395 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-fef652cc-872c-456c-850b-585a72692412 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:33.761395 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:33.761395 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:33.761395 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:33.761395 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:33.761395 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:33.761395 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:33.761395 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:33.761395 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:33.761395 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:33.761395 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:33.761395 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:33.761395 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:33.761395 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:33.761395 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:33.762266 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:33.762266 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:33.762266 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:33.762266 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:33.762266 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:33.762266 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:33.762266 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:33.762266 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:33.762266 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:33.762266 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:33.762266 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:33.762266 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:33.762266 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:33.762266 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:33.762266 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:33.762266 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:33.762266 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:33.763192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:33.763192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:33.763192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:33.763192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:33.763192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 247/511] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:33.763454 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef652cc-872c-456c-850b-585a72692412 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:33.821265 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:33.830387 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 265/512] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:54:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:54:33.832331 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d425354a-616f-41d2-9564-890760208c4e None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:33.873447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef652cc-872c-456c-850b-585a72692412 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:33.907491 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a59f8d-8ed4-422c-848c-4d9b4d34c664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:33.908133 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a59f8d-8ed4-422c-848c-4d9b4d34c664 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:33.908416 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a59f8d-8ed4-422c-848c-4d9b4d34c664 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:33.908689 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a59f8d-8ed4-422c-848c-4d9b4d34c664 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:33.917942 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:33.920174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a59f8d-8ed4-422c-848c-4d9b4d34c664 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:33.921747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a59f8d-8ed4-422c-848c-4d9b4d34c664 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:33.930807 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d425354a-616f-41d2-9564-890760208c4e None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:33.936471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-66a59f8d-8ed4-422c-848c-4d9b4d34c664 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:33.936471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:33.936471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:33.936471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:33.936471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:33.936471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:33.936471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:33.936471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:33.936471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:33.936471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:33.936471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:33.936471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:33.936471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:33.936471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:33.936471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:33.937484 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:33.937484 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:33.937484 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:33.937484 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:33.937484 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:33.937484 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:33.937484 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:33.937484 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:33.937484 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:33.937484 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:33.937484 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:33.937484 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:33.937484 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:33.937484 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:33.937484 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:33.937484 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:33.937484 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:33.938406 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:33.938406 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:33.938406 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:33.938406 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:33.938406 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 248/513] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:33.938664 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a59f8d-8ed4-422c-848c-4d9b4d34c664 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:33.966146 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69187eb6-20fa-465a-9d7b-defa8c9b8987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:33.966756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69187eb6-20fa-465a-9d7b-defa8c9b8987 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:33.967063 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69187eb6-20fa-465a-9d7b-defa8c9b8987 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:33.967334 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69187eb6-20fa-465a-9d7b-defa8c9b8987 None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:33.968013 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69187eb6-20fa-465a-9d7b-defa8c9b8987 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:33.968311 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69187eb6-20fa-465a-9d7b-defa8c9b8987 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:33.970494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69187eb6-20fa-465a-9d7b-defa8c9b8987 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:33.981855 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 266/514] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:33.982473 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69187eb6-20fa-465a-9d7b-defa8c9b8987 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:34.034724 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:34.046117 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a59f8d-8ed4-422c-848c-4d9b4d34c664 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:34.080129 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3057f-83e9-4da8-97f9-058f1be510ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:34.080757 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3057f-83e9-4da8-97f9-058f1be510ed None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:34.081036 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3057f-83e9-4da8-97f9-058f1be510ed None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:34.081126 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69187eb6-20fa-465a-9d7b-defa8c9b8987 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:34.081541 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3057f-83e9-4da8-97f9-058f1be510ed None admin] PATH_INFO: `/v3/users/glance` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:34.088849 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf3057f-83e9-4da8-97f9-058f1be510ed None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:34.090388 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf3057f-83e9-4da8-97f9-058f1be510ed None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:34.102321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-4cf3057f-83e9-4da8-97f9-058f1be510ed None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 03 04:54:34.102321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:34.102321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:34.102321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:34.102321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:34.102321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:34.102321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:34.102321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:34.102321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:34.102321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:34.102321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:34.102321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:34.102321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:54:34.102321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:54:34.102321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:54:34.102321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:54:34.103255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:34.103255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:34.103255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:54:34.103255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:34.103255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:54:34.103255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:34.103255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:34.103255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:34.103255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:34.103255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:34.103255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:34.103255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:34.103255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:34.103255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:34.103255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:34.103255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:34.103255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:34.103255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:34.104174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:34.104174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:34.104174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:54:34.104174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:54:34.104174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:54:34.104174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:54:34.104174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:54:34.104174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:54:34.104174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 03 04:54:34.104174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:34.104174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 249/515] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:54:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:34.104901 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3057f-83e9-4da8-97f9-058f1be510ed None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:34.119462 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505da3b-ac9c-40ba-9bb6-56c0fe74224a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:34.120087 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505da3b-ac9c-40ba-9bb6-56c0fe74224a None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:34.120373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505da3b-ac9c-40ba-9bb6-56c0fe74224a None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:34.120955 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505da3b-ac9c-40ba-9bb6-56c0fe74224a None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:34.121344 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1505da3b-ac9c-40ba-9bb6-56c0fe74224a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:34.121653 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1505da3b-ac9c-40ba-9bb6-56c0fe74224a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:34.123080 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1505da3b-ac9c-40ba-9bb6-56c0fe74224a None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:34.135235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 267/516] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:34.136043 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505da3b-ac9c-40ba-9bb6-56c0fe74224a None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:34.162149 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:34.211812 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3057f-83e9-4da8-97f9-058f1be510ed None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:34.235868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505da3b-ac9c-40ba-9bb6-56c0fe74224a None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:34.244977 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbedca-4668-4bcd-9426-9e00d39042cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:34.245614 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbedca-4668-4bcd-9426-9e00d39042cc None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:34.245899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbedca-4668-4bcd-9426-9e00d39042cc None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:34.246177 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbedca-4668-4bcd-9426-9e00d39042cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:34.249391 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:34.257152 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bbedca-4668-4bcd-9426-9e00d39042cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:34.258627 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bbedca-4668-4bcd-9426-9e00d39042cc None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:34.273608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-39bbedca-4668-4bcd-9426-9e00d39042cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:34.273608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:34.273608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:34.273608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:34.273608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:34.273608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:34.273608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:34.273608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:34.273608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:34.273608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:34.273608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:34.273608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:34.273608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:34.273608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:34.273608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:34.274788 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:34.274788 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:34.274788 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:34.274788 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:34.274788 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:34.274788 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:34.274788 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:34.274788 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:34.274788 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:34.274788 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:34.274788 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:34.274788 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:34.274788 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:34.274788 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:34.274788 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:34.274788 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:34.274788 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:34.276222 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:34.276222 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:34.276222 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:34.276222 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:34.276222 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 250/517] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:34.276222 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbedca-4668-4bcd-9426-9e00d39042cc None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:34.278625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c5ed3a-2902-41d4-931b-b927bc2f085b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:34.279280 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5ed3a-2902-41d4-931b-b927bc2f085b None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:34.279563 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5ed3a-2902-41d4-931b-b927bc2f085b None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:34.280486 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5ed3a-2902-41d4-931b-b927bc2f085b None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:34.280600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c5ed3a-2902-41d4-931b-b927bc2f085b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:34.280821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c5ed3a-2902-41d4-931b-b927bc2f085b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:34.282790 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c5ed3a-2902-41d4-931b-b927bc2f085b None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:34.294599 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 268/518] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:34.295441 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c5ed3a-2902-41d4-931b-b927bc2f085b None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:34.379826 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbedca-4668-4bcd-9426-9e00d39042cc None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:34.382274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:34.395686 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c5ed3a-2902-41d4-931b-b927bc2f085b None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:34.414404 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087802d-49a9-4211-ad38-ca588d2d5587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:34.415133 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087802d-49a9-4211-ad38-ca588d2d5587 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:34.415422 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087802d-49a9-4211-ad38-ca588d2d5587 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:34.415701 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087802d-49a9-4211-ad38-ca588d2d5587 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:34.422599 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8087802d-49a9-4211-ad38-ca588d2d5587 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:34.424091 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8087802d-49a9-4211-ad38-ca588d2d5587 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:34.429039 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500e2e6-9f7a-4914-9f34-91fb8a700c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:34.429656 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500e2e6-9f7a-4914-9f34-91fb8a700c45 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:34.429943 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500e2e6-9f7a-4914-9f34-91fb8a700c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:34.430224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500e2e6-9f7a-4914-9f34-91fb8a700c45 None admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:34.431047 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500e2e6-9f7a-4914-9f34-91fb8a700c45 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:34.431363 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500e2e6-9f7a-4914-9f34-91fb8a700c45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:34.432794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500e2e6-9f7a-4914-9f34-91fb8a700c45 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:34.435810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-8087802d-49a9-4211-ad38-ca588d2d5587 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:34.435810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:34.435810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:34.435810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:34.435810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:34.435810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:34.435810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:34.435810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:34.435810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:34.435810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:34.435810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:34.435810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:34.435810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:54:34.435810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:54:34.435810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:54:34.436890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:54:34.436890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:34.436890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:34.436890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:34.436890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:34.436890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:34.436890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:34.436890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:34.436890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:34.436890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:34.436890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:34.436890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:34.436890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:34.436890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:34.436890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:34.436890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:34.436890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:34.438339 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:54:34.438339 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:54:34.438339 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:54:34.438339 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:54:34.438339 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:54:34.438339 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:54:34.438339 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:34.438339 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:34.438339 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 251/519] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:54:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:34.439629 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087802d-49a9-4211-ad38-ca588d2d5587 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:34.450559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 269/520] 10.222.0.40 () {56 vars in 1137 bytes} [Wed Aug 3 04:54:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:34.451433 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500e2e6-9f7a-4914-9f34-91fb8a700c45 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:34.481598 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:34.541772 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087802d-49a9-4211-ad38-ca588d2d5587 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:34.550856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500e2e6-9f7a-4914-9f34-91fb8a700c45 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:34.575269 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02088b40-093b-4d33-9826-cc6079ddb215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:34.575988 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02088b40-093b-4d33-9826-cc6079ddb215 None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:34.576276 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02088b40-093b-4d33-9826-cc6079ddb215 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:34.576551 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02088b40-093b-4d33-9826-cc6079ddb215 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:34.577211 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02088b40-093b-4d33-9826-cc6079ddb215 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:34.579205 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02088b40-093b-4d33-9826-cc6079ddb215 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:34.585671 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b4419-dc66-42d1-ab22-fdd6d41199af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:34.586294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b4419-dc66-42d1-ab22-fdd6d41199af None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:34.586580 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b4419-dc66-42d1-ab22-fdd6d41199af None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:34.586882 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b4419-dc66-42d1-ab22-fdd6d41199af None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:34.587570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6b4419-dc66-42d1-ab22-fdd6d41199af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:34.587888 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6b4419-dc66-42d1-ab22-fdd6d41199af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:34.589252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6b4419-dc66-42d1-ab22-fdd6d41199af None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:34.600547 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 270/521] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:34.601333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b4419-dc66-42d1-ab22-fdd6d41199af None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:34.654603 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 252/522] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 3 04:54:34 2022] POST /identity/v3/users => generated 247 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:54:34.655455 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02088b40-093b-4d33-9826-cc6079ddb215 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:34.682017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:34.715775 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:34.729929 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b4419-dc66-42d1-ab22-fdd6d41199af None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:34.771245 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38bb737-7396-445f-83b1-c397512a81f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:34.771907 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bb737-7396-445f-83b1-c397512a81f0 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:34.772013 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02088b40-093b-4d33-9826-cc6079ddb215 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:34.772191 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bb737-7396-445f-83b1-c397512a81f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:34.772470 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bb737-7396-445f-83b1-c397512a81f0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:34.782360 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38bb737-7396-445f-83b1-c397512a81f0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:34.784036 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38bb737-7396-445f-83b1-c397512a81f0 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:34.797807 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-d38bb737-7396-445f-83b1-c397512a81f0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 03 04:54:34.797807 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:34.797807 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:34.797807 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:34.797807 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:34.797807 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:34.797807 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:34.797807 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:34.797807 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:34.797807 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:34.797807 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:34.797807 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:34.797807 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:54:34.797807 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:54:34.797807 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:54:34.797807 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:54:34.798774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:34.798774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:34.798774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:54:34.798774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:34.798774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:54:34.798774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:34.798774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:34.798774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:34.798774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:34.798774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:34.798774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:34.798774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:34.798774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:34.798774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:34.798774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:34.798774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:34.798774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:34.798774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:34.799734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:34.799734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:34.799734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:54:34.799734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:54:34.799734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:54:34.799734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:54:34.799734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:54:34.799734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:54:34.799734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 03 04:54:34.799734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:34.799734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 271/523] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 3 04:54:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:34.800296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38bb737-7396-445f-83b1-c397512a81f0 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:34.819190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933b7d7-5aac-416f-9ebc-266bb30602e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:34.819828 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933b7d7-5aac-416f-9ebc-266bb30602e8 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:34.820166 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933b7d7-5aac-416f-9ebc-266bb30602e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:34.820386 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933b7d7-5aac-416f-9ebc-266bb30602e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:34.821146 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4933b7d7-5aac-416f-9ebc-266bb30602e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:34.821454 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4933b7d7-5aac-416f-9ebc-266bb30602e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:34.823002 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4933b7d7-5aac-416f-9ebc-266bb30602e8 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:34.844129 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 253/524] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:54:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:34.844987 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933b7d7-5aac-416f-9ebc-266bb30602e8 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:34.913428 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38bb737-7396-445f-83b1-c397512a81f0 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:34.943703 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:34.954048 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd8faa-6a28-4c1a-b5fe-e31759f7cb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:34.954812 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd8faa-6a28-4c1a-b5fe-e31759f7cb2b None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:34.955007 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd8faa-6a28-4c1a-b5fe-e31759f7cb2b None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:34.955243 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd8faa-6a28-4c1a-b5fe-e31759f7cb2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:34.963425 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933b7d7-5aac-416f-9ebc-266bb30602e8 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:34.975127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:35.002275 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1b1e4-fa26-4976-a139-3ce981cbabd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:35.002949 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1b1e4-fa26-4976-a139-3ce981cbabd2 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:35.003236 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1b1e4-fa26-4976-a139-3ce981cbabd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:35.003514 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1b1e4-fa26-4976-a139-3ce981cbabd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:35.010361 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d1b1e4-fa26-4976-a139-3ce981cbabd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:35.011853 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d1b1e4-fa26-4976-a139-3ce981cbabd2 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:35.023296 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-55d1b1e4-fa26-4976-a139-3ce981cbabd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:35.023296 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:35.023296 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:35.023296 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:35.023296 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:35.023296 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:35.023296 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:35.023296 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:35.023296 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:35.023296 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:35.023296 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:35.023296 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:35.023296 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:54:35.023296 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:54:35.023296 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:54:35.024453 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:54:35.024453 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:35.024453 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:35.024453 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:35.024453 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:35.024453 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:35.024453 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:35.024453 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:35.024453 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:35.024453 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:35.024453 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:35.024453 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:35.024453 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:35.024453 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:35.024453 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:35.024453 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:35.024453 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:35.025312 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:54:35.025312 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:54:35.025312 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:54:35.025312 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:54:35.025312 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:54:35.025312 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:54:35.025312 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:35.025312 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:35.025312 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 254/525] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:54:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:35.025861 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1b1e4-fa26-4976-a139-3ce981cbabd2 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:35.032901 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dd8faa-6a28-4c1a-b5fe-e31759f7cb2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:35.034381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dd8faa-6a28-4c1a-b5fe-e31759f7cb2b None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:35.132992 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1b1e4-fa26-4976-a139-3ce981cbabd2 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:35.143383 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 272/526] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:54:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:54:35.144228 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd8faa-6a28-4c1a-b5fe-e31759f7cb2b None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:35.175024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff1617-d093-4603-9dc1-7876101189f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:35.175705 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff1617-d093-4603-9dc1-7876101189f0 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:35.175951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff1617-d093-4603-9dc1-7876101189f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:35.176210 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff1617-d093-4603-9dc1-7876101189f0 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:35.177015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aff1617-d093-4603-9dc1-7876101189f0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:35.177326 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aff1617-d093-4603-9dc1-7876101189f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:35.178770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aff1617-d093-4603-9dc1-7876101189f0 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:35.188547 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:35.196831 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 255/527] 10.222.0.40 () {56 vars in 1139 bytes} [Wed Aug 3 04:54:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:35.197803 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff1617-d093-4603-9dc1-7876101189f0 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:35.204920 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:35.264937 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd8faa-6a28-4c1a-b5fe-e31759f7cb2b None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:35.303466 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c73aa42-a53a-4397-97da-704002f54f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:35.303982 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73aa42-a53a-4397-97da-704002f54f51 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:35.304257 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73aa42-a53a-4397-97da-704002f54f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:35.304257 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73aa42-a53a-4397-97da-704002f54f51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:35.307050 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c73aa42-a53a-4397-97da-704002f54f51 None admin] RBAC: Adding query filter params (role.id=cae571342db14f5fac7849efe7f8636e, scope.project.id=cf3763a37383456184a9ca9351b4ce03, user.id=c81ca55d52244bf1bfe4e829b6880c20) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:35.307050 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c73aa42-a53a-4397-97da-704002f54f51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:35.307211 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c73aa42-a53a-4397-97da-704002f54f51 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:35.310882 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff1617-d093-4603-9dc1-7876101189f0 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:35.314630 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 273/528] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:54:35 2022] GET /identity/v3/role_assignments?user.id=c81ca55d52244bf1bfe4e829b6880c20&scope.project.id=cf3763a37383456184a9ca9351b4ce03&role.id=cae571342db14f5fac7849efe7f8636e => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:35.315325 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c73aa42-a53a-4397-97da-704002f54f51 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:35.366049 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c80dd4-69d7-4c2d-a12c-dd13c551d8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:35.367496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c80dd4-69d7-4c2d-a12c-dd13c551d8bf None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:35.369046 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c80dd4-69d7-4c2d-a12c-dd13c551d8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:35.369682 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c80dd4-69d7-4c2d-a12c-dd13c551d8bf None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:35.370736 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c80dd4-69d7-4c2d-a12c-dd13c551d8bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:35.371306 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c80dd4-69d7-4c2d-a12c-dd13c551d8bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:35.373059 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c80dd4-69d7-4c2d-a12c-dd13c551d8bf None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:35.395148 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 256/529] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:54:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:35.397135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c80dd4-69d7-4c2d-a12c-dd13c551d8bf None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:35.449744 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c73aa42-a53a-4397-97da-704002f54f51 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:35.465075 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:35.479958 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:35.495809 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c95d1-f8d9-4c9a-b293-b564851798ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:35.496469 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c95d1-f8d9-4c9a-b293-b564851798ac None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:35.496755 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c95d1-f8d9-4c9a-b293-b564851798ac None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:35.497516 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c95d1-f8d9-4c9a-b293-b564851798ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:35.512616 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349c95d1-f8d9-4c9a-b293-b564851798ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:35.514284 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349c95d1-f8d9-4c9a-b293-b564851798ac None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:35.523396 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c80dd4-69d7-4c2d-a12c-dd13c551d8bf None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:35.533575 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-349c95d1-f8d9-4c9a-b293-b564851798ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:35.533575 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:35.533575 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:35.533575 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:35.533575 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:35.533575 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:35.533575 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:35.533575 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:35.533575 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:35.533575 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:35.533575 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:35.533575 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:35.533575 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:35.533575 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:35.533575 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:35.534649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:35.534649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:35.534649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:35.534649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:35.534649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:35.534649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:35.534649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:35.534649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:35.534649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:35.534649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:35.534649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:35.534649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:35.534649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:35.534649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:35.534649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:35.534649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:35.534649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:35.535740 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:35.535740 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:35.535740 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:35.535740 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:35.537814 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 274/530] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:35.539293 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c95d1-f8d9-4c9a-b293-b564851798ac None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:35.564714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a204f-cd4a-47c7-b246-315736f95e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:35.565507 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a204f-cd4a-47c7-b246-315736f95e9c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:35.565880 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a204f-cd4a-47c7-b246-315736f95e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:35.566347 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a204f-cd4a-47c7-b246-315736f95e9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:35.645851 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a204f-cd4a-47c7-b246-315736f95e9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:35.647561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a204f-cd4a-47c7-b246-315736f95e9c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:35.686407 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c95d1-f8d9-4c9a-b293-b564851798ac None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:35.725264 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f46e0-bd11-44e3-8a79-397f59a8b6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:35.726069 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f46e0-bd11-44e3-8a79-397f59a8b6ec None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:35.726428 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f46e0-bd11-44e3-8a79-397f59a8b6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:35.726753 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f46e0-bd11-44e3-8a79-397f59a8b6ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:35.727673 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f46e0-bd11-44e3-8a79-397f59a8b6ec None admin] RBAC: Adding query filter params (role.id=cae571342db14f5fac7849efe7f8636e, scope.project.id=cf3763a37383456184a9ca9351b4ce03, user.id=a3fa4a6c16274c18849869b55b33ac62) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:35.728082 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f46e0-bd11-44e3-8a79-397f59a8b6ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:35.728314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:35.730082 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f46e0-bd11-44e3-8a79-397f59a8b6ec None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:35.737044 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 275/531] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:54:35 2022] GET /identity/v3/role_assignments?user.id=a3fa4a6c16274c18849869b55b33ac62&scope.project.id=cf3763a37383456184a9ca9351b4ce03&role.id=cae571342db14f5fac7849efe7f8636e => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:35.738161 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f46e0-bd11-44e3-8a79-397f59a8b6ec None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:35.779278 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:35.793653 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 257/532] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:54:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:54:35.803992 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a204f-cd4a-47c7-b246-315736f95e9c None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:35.861905 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f46e0-bd11-44e3-8a79-397f59a8b6ec None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:35.908719 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c2da7-c0a4-4f89-a15e-b088ea5040db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:35.909410 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c2da7-c0a4-4f89-a15e-b088ea5040db None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:35.909696 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c2da7-c0a4-4f89-a15e-b088ea5040db None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:35.909966 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c2da7-c0a4-4f89-a15e-b088ea5040db None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:35.910664 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c2da7-c0a4-4f89-a15e-b088ea5040db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:35.914110 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c2da7-c0a4-4f89-a15e-b088ea5040db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:35.914110 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c2da7-c0a4-4f89-a15e-b088ea5040db None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:35.938588 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 276/533] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:35.942399 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c2da7-c0a4-4f89-a15e-b088ea5040db None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:35.987895 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a204f-cd4a-47c7-b246-315736f95e9c None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:36.058986 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f4109-ca08-4c0c-98ed-240c16b25488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:36.059639 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f4109-ca08-4c0c-98ed-240c16b25488 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:36.059917 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f4109-ca08-4c0c-98ed-240c16b25488 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:36.060190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f4109-ca08-4c0c-98ed-240c16b25488 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:36.082493 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f4109-ca08-4c0c-98ed-240c16b25488 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:36.084037 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f4109-ca08-4c0c-98ed-240c16b25488 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:36.115287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:36.115905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-911f4109-ca08-4c0c-98ed-240c16b25488 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:36.115905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:36.115905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:36.115905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:36.115905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:36.115905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:36.115905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:36.115905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:36.115905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:36.115905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:36.115905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:36.115905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:36.115905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:36.115905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:36.115905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:36.116753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:36.116753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:36.116753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:36.116753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:36.116753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:36.116753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:36.116753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:36.116753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:36.116753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:36.116753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:36.116753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:36.116753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:36.116753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:36.116753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:36.116753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:36.116753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:36.116753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:36.117611 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:36.117611 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:36.117611 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:36.117611 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:36.123022 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 258/534] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:36.125119 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c2da7-c0a4-4f89-a15e-b088ea5040db None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:36.127574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f4109-ca08-4c0c-98ed-240c16b25488 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:36.150285 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:36.195197 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ce626-a34e-4174-bc38-395d4bb3ee46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:36.195877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ce626-a34e-4174-bc38-395d4bb3ee46 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:36.196101 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ce626-a34e-4174-bc38-395d4bb3ee46 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:36.196372 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ce626-a34e-4174-bc38-395d4bb3ee46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:36.210975 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865ce626-a34e-4174-bc38-395d4bb3ee46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:36.213869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865ce626-a34e-4174-bc38-395d4bb3ee46 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:36.242030 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-865ce626-a34e-4174-bc38-395d4bb3ee46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:36.242030 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:36.242030 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:36.242030 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:36.242030 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:36.242030 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:36.242030 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:36.242030 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:36.242030 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:36.242030 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:36.242030 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:36.242030 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:36.242030 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:54:36.242030 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:54:36.242030 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:54:36.244094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:54:36.244094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:36.244094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:36.244094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:36.244094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:36.244094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:36.244094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:36.244094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:36.244094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:36.244094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:36.244094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:36.244094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:36.244094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:36.244094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:36.244094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:36.244094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:36.244094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:36.246129 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:54:36.246129 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:54:36.246129 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:54:36.246129 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:54:36.246129 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:54:36.246129 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:54:36.246129 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:36.246129 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:36.247101 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 277/535] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:54:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:36.252688 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ce626-a34e-4174-bc38-395d4bb3ee46 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:36.305875 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f4109-ca08-4c0c-98ed-240c16b25488 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:36.373540 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99baa3e1-ac0e-4e37-bb6a-2795031a6a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:36.374113 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99baa3e1-ac0e-4e37-bb6a-2795031a6a84 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:36.374395 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99baa3e1-ac0e-4e37-bb6a-2795031a6a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:36.374770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99baa3e1-ac0e-4e37-bb6a-2795031a6a84 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:36.375481 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99baa3e1-ac0e-4e37-bb6a-2795031a6a84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:36.375811 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99baa3e1-ac0e-4e37-bb6a-2795031a6a84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:36.377188 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99baa3e1-ac0e-4e37-bb6a-2795031a6a84 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:36.408571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 259/536] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:36.414351 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99baa3e1-ac0e-4e37-bb6a-2795031a6a84 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:36.431273 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ce626-a34e-4174-bc38-395d4bb3ee46 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:36.486290 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b963ece-e807-4e00-a386-906651290eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:36.486954 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b963ece-e807-4e00-a386-906651290eed None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:36.487235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b963ece-e807-4e00-a386-906651290eed None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:36.487510 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b963ece-e807-4e00-a386-906651290eed None admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:36.488263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b963ece-e807-4e00-a386-906651290eed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:36.488565 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b963ece-e807-4e00-a386-906651290eed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:36.490015 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b963ece-e807-4e00-a386-906651290eed None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:36.532769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 278/537] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:54:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:36.543876 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b963ece-e807-4e00-a386-906651290eed None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:36.549643 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:36.567195 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:36.587239 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99baa3e1-ac0e-4e37-bb6a-2795031a6a84 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:36.661302 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6305d600-5573-4b2c-b94d-2898e399acac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:36.662282 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305d600-5573-4b2c-b94d-2898e399acac None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:36.662767 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305d600-5573-4b2c-b94d-2898e399acac None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:36.663263 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305d600-5573-4b2c-b94d-2898e399acac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:36.672176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6305d600-5573-4b2c-b94d-2898e399acac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:36.673869 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6305d600-5573-4b2c-b94d-2898e399acac None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:36.686651 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-6305d600-5573-4b2c-b94d-2898e399acac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:36.686651 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:36.686651 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:36.686651 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:36.686651 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:36.686651 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:36.686651 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:36.686651 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:36.686651 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:36.686651 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:36.686651 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:36.686651 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:36.686651 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:54:36.686651 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:54:36.686651 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:54:36.688068 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:54:36.688068 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:36.688068 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:36.688068 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:36.688068 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:36.688068 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:36.688068 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:36.688068 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:36.688068 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:36.688068 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:36.688068 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:36.688068 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:36.688068 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:36.688068 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:36.688068 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:36.688068 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:36.688068 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:36.688893 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:54:36.688893 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:54:36.688893 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:54:36.688893 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:54:36.688893 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:54:36.688893 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:54:36.688893 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:36.688893 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:36.691586 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 260/538] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:54:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:54:36.696837 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6305d600-5573-4b2c-b94d-2898e399acac None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:36.796843 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b963ece-e807-4e00-a386-906651290eed None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:36.812922 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6305d600-5573-4b2c-b94d-2898e399acac None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:36.857921 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:36.866248 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565e3c9-f32d-4046-9daf-5aebf16cf707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:36.867753 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565e3c9-f32d-4046-9daf-5aebf16cf707 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:36.868607 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f14c30e-11ab-4b71-9c26-d1b95abc7c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:36.869168 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565e3c9-f32d-4046-9daf-5aebf16cf707 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:36.869972 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14c30e-11ab-4b71-9c26-d1b95abc7c47 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:36.870435 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565e3c9-f32d-4046-9daf-5aebf16cf707 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:36.871411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5565e3c9-f32d-4046-9daf-5aebf16cf707 None admin] RBAC: Adding query filter params (role.id=cae571342db14f5fac7849efe7f8636e, scope.project.id=cf3763a37383456184a9ca9351b4ce03, user.id=84cc14b33895443b893fab487bfe1858) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:36.876855 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14c30e-11ab-4b71-9c26-d1b95abc7c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:36.876855 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14c30e-11ab-4b71-9c26-d1b95abc7c47 None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:36.876855 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f14c30e-11ab-4b71-9c26-d1b95abc7c47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:36.876855 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f14c30e-11ab-4b71-9c26-d1b95abc7c47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:36.876855 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f14c30e-11ab-4b71-9c26-d1b95abc7c47 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:36.877196 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5565e3c9-f32d-4046-9daf-5aebf16cf707 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:36.878221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5565e3c9-f32d-4046-9daf-5aebf16cf707 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:36.892053 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 279/539] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:54:36 2022] GET /identity/v3/role_assignments?user.id=84cc14b33895443b893fab487bfe1858&scope.project.id=cf3763a37383456184a9ca9351b4ce03&role.id=cae571342db14f5fac7849efe7f8636e => generated 259 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:36.908255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 261/540] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:54:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:36.914175 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565e3c9-f32d-4046-9daf-5aebf16cf707 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:36.994679 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ccd58-cd3c-40d0-b823-c9037f7d0669 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:36.995565 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ccd58-cd3c-40d0-b823-c9037f7d0669 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:36.996343 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ccd58-cd3c-40d0-b823-c9037f7d0669 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:36.997892 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 262/541] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:37.005633 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7d1a8-7b7b-4553-b8bf-1bb5b4346ce3 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:37.006325 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7d1a8-7b7b-4553-b8bf-1bb5b4346ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:37.007067 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7d1a8-7b7b-4553-b8bf-1bb5b4346ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:37.051258 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:37.052059 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-b9a7d1a8-7b7b-4553-b8bf-1bb5b4346ce3 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:37.053334 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b9a7d1a8-7b7b-4553-b8bf-1bb5b4346ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:37.076336 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565e3c9-f32d-4046-9daf-5aebf16cf707 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:37.117160 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32000166-0c4b-4e89-b537-5bd4a09b5139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:37.117922 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32000166-0c4b-4e89-b537-5bd4a09b5139 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:37.118284 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32000166-0c4b-4e89-b537-5bd4a09b5139 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:37.118638 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32000166-0c4b-4e89-b537-5bd4a09b5139 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:37.119586 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32000166-0c4b-4e89-b537-5bd4a09b5139 None admin] RBAC: Adding query filter params (role.id=cae571342db14f5fac7849efe7f8636e, scope.project.id=cf3763a37383456184a9ca9351b4ce03, user.id=cff15ea4cbc64b068aa5f6586b5bff87) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:37.119976 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32000166-0c4b-4e89-b537-5bd4a09b5139 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:37.122591 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32000166-0c4b-4e89-b537-5bd4a09b5139 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:37.130313 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 280/542] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:54:36 2022] GET /identity/v3/role_assignments?user.id=cff15ea4cbc64b068aa5f6586b5bff87&scope.project.id=cf3763a37383456184a9ca9351b4ce03&role.id=cae571342db14f5fac7849efe7f8636e => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:37.192311 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 263/543] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:37.215772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32000166-0c4b-4e89-b537-5bd4a09b5139 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:37.369548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32000166-0c4b-4e89-b537-5bd4a09b5139 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:37.423869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59853b9-f782-479f-a745-bcb897a69815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:37.424755 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59853b9-f782-479f-a745-bcb897a69815 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:37.425207 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59853b9-f782-479f-a745-bcb897a69815 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:37.425636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59853b9-f782-479f-a745-bcb897a69815 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:37.438829 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:37.447656 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59853b9-f782-479f-a745-bcb897a69815 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:37.453245 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-f59853b9-f782-479f-a745-bcb897a69815 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:37.453245 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:37.453245 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:37.453245 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:37.453245 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:37.453245 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:37.453245 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:37.453245 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:37.453245 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:37.453245 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:37.453245 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:37.453245 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:37.453245 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:37.453245 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:37.453245 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:37.453245 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:37.455069 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:37.455069 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:37.455069 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:37.455069 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:37.455069 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:37.455069 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:37.455069 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:37.455069 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:37.455069 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:37.455069 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:37.455069 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:37.455069 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:37.455069 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:37.455069 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:37.455069 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:37.455069 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:37.455865 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:37.455865 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:37.455865 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:37.455865 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:37.455865 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:37.455865 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:37.455865 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:37.455865 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:37.455865 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:37.455865 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:37.455865 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:37.455865 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:37.455865 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 281/544] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 3 04:54:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:37.462789 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7d1a8-7b7b-4553-b8bf-1bb5b4346ce3 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:37.472405 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de4c5b-1b3b-47ee-83bb-7ac67e62439e None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:37.473193 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de4c5b-1b3b-47ee-83bb-7ac67e62439e None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:37.473885 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de4c5b-1b3b-47ee-83bb-7ac67e62439e None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:37.475988 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 282/545] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:37.481494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437aefaa-2c5a-43c8-add5-1a924e93b5e2 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:37.482052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437aefaa-2c5a-43c8-add5-1a924e93b5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:37.482533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437aefaa-2c5a-43c8-add5-1a924e93b5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:37.514361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-437aefaa-2c5a-43c8-add5-1a924e93b5e2 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:37.517223 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-437aefaa-2c5a-43c8-add5-1a924e93b5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:37.614975 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 283/546] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:37.645476 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7d1a8-7b7b-4553-b8bf-1bb5b4346ce3 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:37.666594 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437aefaa-2c5a-43c8-add5-1a924e93b5e2 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:37.684148 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:37.715476 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476ee65-0e7e-44ab-9fb6-cde08d8876c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:37.715476 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476ee65-0e7e-44ab-9fb6-cde08d8876c2 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:37.715476 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476ee65-0e7e-44ab-9fb6-cde08d8876c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:37.715476 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476ee65-0e7e-44ab-9fb6-cde08d8876c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:37.716405 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476ee65-0e7e-44ab-9fb6-cde08d8876c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:37.723318 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476ee65-0e7e-44ab-9fb6-cde08d8876c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:37.724758 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476ee65-0e7e-44ab-9fb6-cde08d8876c2 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:37.738052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 264/547] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 3 04:54:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:37.749462 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476ee65-0e7e-44ab-9fb6-cde08d8876c2 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:37.815486 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437aefaa-2c5a-43c8-add5-1a924e93b5e2 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:37.867857 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:37.882096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a55ab39-9fb1-4d71-95c3-934ad69c039d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:37.886665 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55ab39-9fb1-4d71-95c3-934ad69c039d None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:37.886665 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55ab39-9fb1-4d71-95c3-934ad69c039d None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:37.886665 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55ab39-9fb1-4d71-95c3-934ad69c039d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:37.905480 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a55ab39-9fb1-4d71-95c3-934ad69c039d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:37.919351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-1a55ab39-9fb1-4d71-95c3-934ad69c039d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:37.919351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:37.919351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:37.919351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:37.919351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:37.919351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:37.919351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:37.919351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:37.919351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:37.919351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:37.919351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:37.919351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:37.919351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:37.919351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:37.919351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:37.919351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:37.920276 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:37.920276 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:37.920276 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:37.920276 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:37.920276 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:37.920276 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:37.920276 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:37.920276 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:37.920276 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:37.920276 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:37.920276 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:37.920276 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:37.920276 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:37.920276 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:37.920276 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:37.920276 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:37.921168 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:37.921168 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:37.921168 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:37.921168 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:37.921168 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:37.921168 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:37.921168 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:37.921168 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:37.921168 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:37.921168 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:37.921168 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:37.921168 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:37.921168 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 284/548] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 3 04:54:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:54:37.926388 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a55ab39-9fb1-4d71-95c3-934ad69c039d None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:37.991952 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476ee65-0e7e-44ab-9fb6-cde08d8876c2 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:38.045932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a55ab39-9fb1-4d71-95c3-934ad69c039d None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:38.081152 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed551a1-d6f1-4288-b025-e207ca2d1848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:38.081811 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed551a1-d6f1-4288-b025-e207ca2d1848 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:38.082090 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed551a1-d6f1-4288-b025-e207ca2d1848 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:38.082349 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed551a1-d6f1-4288-b025-e207ca2d1848 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:38.096354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa36886-a8ec-4021-9420-4e676cd07ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:38.097359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa36886-a8ec-4021-9420-4e676cd07ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:38.097847 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa36886-a8ec-4021-9420-4e676cd07ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:38.098332 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa36886-a8ec-4021-9420-4e676cd07ca9 None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:38.099300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa36886-a8ec-4021-9420-4e676cd07ca9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:38.099810 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa36886-a8ec-4021-9420-4e676cd07ca9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:38.101414 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa36886-a8ec-4021-9420-4e676cd07ca9 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:38.105032 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed551a1-d6f1-4288-b025-e207ca2d1848 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:38.106716 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed551a1-d6f1-4288-b025-e207ca2d1848 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:38.118941 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 285/549] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 3 04:54:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:38.125656 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa36886-a8ec-4021-9420-4e676cd07ca9 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:38.146410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-8ed551a1-d6f1-4288-b025-e207ca2d1848 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:38.146410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:38.146410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:38.146410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:38.146410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:38.146410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:38.146410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:38.146410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:38.146410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:38.146410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:38.146410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:38.146410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:38.146410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:38.146410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:38.146410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:38.147497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:38.147497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:38.147497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:38.147497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:38.147497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:38.147497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:38.147497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:38.147497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:38.147497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:38.147497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:38.147497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:38.147497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:38.147497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:38.147497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:38.147497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:38.147497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:38.147497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:38.148588 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:38.148588 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:38.148588 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:38.148588 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:38.149541 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 265/550] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:38.150931 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32cc9a-a508-485c-8481-f3e278fedb21 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:38.151321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32cc9a-a508-485c-8481-f3e278fedb21 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:38.151704 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32cc9a-a508-485c-8481-f3e278fedb21 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:38.157018 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 266/551] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:38.157926 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32cc9a-a508-485c-8481-f3e278fedb21 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:38.197060 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:38.214224 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:38.267667 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa36886-a8ec-4021-9420-4e676cd07ca9 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:38.277182 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32cc9a-a508-485c-8481-f3e278fedb21 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:38.312677 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e4bba-4e4b-4676-bb45-f43020b25278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:38.315424 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e4bba-4e4b-4676-bb45-f43020b25278 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:38.315424 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e4bba-4e4b-4676-bb45-f43020b25278 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:38.315424 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e4bba-4e4b-4676-bb45-f43020b25278 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:38.320516 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0833e2-ff60-4161-a8c2-d0b72cbec4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:38.321384 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0833e2-ff60-4161-a8c2-d0b72cbec4f5 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:38.321670 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0833e2-ff60-4161-a8c2-d0b72cbec4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:38.322052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0833e2-ff60-4161-a8c2-d0b72cbec4f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:38.322941 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0833e2-ff60-4161-a8c2-d0b72cbec4f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:38.323356 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0833e2-ff60-4161-a8c2-d0b72cbec4f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:38.325415 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0833e2-ff60-4161-a8c2-d0b72cbec4f5 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:38.329981 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e4bba-4e4b-4676-bb45-f43020b25278 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:38.331728 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e4bba-4e4b-4676-bb45-f43020b25278 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:38.344896 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 267/552] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:38.344896 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64ee0b-2aec-489c-b439-b9539b4e1629 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:38.344896 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64ee0b-2aec-489c-b439-b9539b4e1629 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:38.344896 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64ee0b-2aec-489c-b439-b9539b4e1629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:38.356602 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-f95e4bba-4e4b-4676-bb45-f43020b25278 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:38.356602 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:38.356602 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:38.356602 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:38.356602 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:38.356602 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:38.356602 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:38.356602 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:38.356602 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:38.356602 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:38.356602 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:38.356602 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:38.356602 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:38.356602 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:38.356602 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:38.357574 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:38.357574 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:38.357574 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:38.357574 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:38.357574 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:38.357574 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:38.357574 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:38.357574 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:38.357574 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:38.357574 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:38.357574 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:38.357574 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:38.357574 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:38.357574 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:38.357574 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:38.357574 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:38.357574 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:38.358491 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:38.358491 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:38.358491 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:38.358491 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:38.360952 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 286/553] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:38.362024 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e4bba-4e4b-4676-bb45-f43020b25278 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:38.367433 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-8b64ee0b-2aec-489c-b439-b9539b4e1629 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:38.372324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8b64ee0b-2aec-489c-b439-b9539b4e1629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:38.466926 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 268/554] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:38.468473 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64ee0b-2aec-489c-b439-b9539b4e1629 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:38.505631 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:38.509290 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e4bba-4e4b-4676-bb45-f43020b25278 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:38.511040 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:38.548495 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085ab62-0692-40eb-8b32-094a82c42f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:38.549230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085ab62-0692-40eb-8b32-094a82c42f40 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:38.549600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085ab62-0692-40eb-8b32-094a82c42f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:38.549980 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085ab62-0692-40eb-8b32-094a82c42f40 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:38.563563 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085ab62-0692-40eb-8b32-094a82c42f40 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:38.565237 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085ab62-0692-40eb-8b32-094a82c42f40 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:38.581284 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-2085ab62-0692-40eb-8b32-094a82c42f40 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 03 04:54:38.581284 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:38.581284 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:38.581284 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:38.581284 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:38.581284 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:38.581284 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:38.581284 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:38.581284 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:38.581284 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:38.581284 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:38.581284 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:38.581284 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:54:38.581284 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:54:38.581284 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:54:38.581284 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:54:38.582310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:38.582310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:38.582310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:54:38.582310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:38.582310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:54:38.582310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:38.582310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:38.582310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:38.582310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:38.582310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:38.582310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:38.582310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:38.582310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:38.582310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:38.582310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:38.582310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:38.582310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:38.582310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:38.583656 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:38.583656 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:38.583656 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:54:38.583656 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:54:38.583656 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:54:38.583656 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:54:38.583656 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:54:38.583656 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:54:38.583656 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 03 04:54:38.583656 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:38.587300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 287/555] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:54:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:38.588298 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085ab62-0692-40eb-8b32-094a82c42f40 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:38.596772 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64ee0b-2aec-489c-b439-b9539b4e1629 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:38.636245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89827d8-8f88-484d-bb22-516adcf7b460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:38.637195 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89827d8-8f88-484d-bb22-516adcf7b460 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:38.637705 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89827d8-8f88-484d-bb22-516adcf7b460 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:38.638226 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89827d8-8f88-484d-bb22-516adcf7b460 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:38.639241 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89827d8-8f88-484d-bb22-516adcf7b460 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:38.639859 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89827d8-8f88-484d-bb22-516adcf7b460 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:38.641632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89827d8-8f88-484d-bb22-516adcf7b460 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:38.654683 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 269/556] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:38.655955 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89827d8-8f88-484d-bb22-516adcf7b460 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:38.712760 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085ab62-0692-40eb-8b32-094a82c42f40 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:38.747969 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b69a9-539d-4df1-8893-7540965c625d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:38.748980 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b69a9-539d-4df1-8893-7540965c625d None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:38.749516 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b69a9-539d-4df1-8893-7540965c625d None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:38.750506 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b69a9-539d-4df1-8893-7540965c625d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:38.757275 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:38.761581 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b69a9-539d-4df1-8893-7540965c625d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:38.767586 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-df5b69a9-539d-4df1-8893-7540965c625d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:38.767586 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:38.767586 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:38.767586 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:38.767586 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:38.767586 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:38.767586 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:38.767586 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:38.767586 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:38.767586 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:38.767586 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:38.767586 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:38.767586 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:38.767586 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:38.767586 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:38.767586 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:38.769835 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:38.769835 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:38.769835 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:38.769835 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:38.769835 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:38.769835 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:38.769835 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:38.769835 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:38.769835 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:38.769835 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:38.769835 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:38.769835 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:38.769835 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:38.769835 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:38.769835 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:38.769835 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:38.771167 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:38.771167 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:38.771167 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:38.771167 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:38.771167 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:38.771167 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:38.771167 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:38.771167 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:38.771167 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:38.771167 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:38.771167 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:38.771167 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:38.772714 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 288/557] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 3 04:54:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:38.774165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b69a9-539d-4df1-8893-7540965c625d None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:38.779305 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89827d8-8f88-484d-bb22-516adcf7b460 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:38.801807 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:38.820770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f2c34-65b3-4dcd-9b88-13a6739dfb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:38.821514 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f2c34-65b3-4dcd-9b88-13a6739dfb01 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:38.821729 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f2c34-65b3-4dcd-9b88-13a6739dfb01 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:38.822008 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f2c34-65b3-4dcd-9b88-13a6739dfb01 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:38.822892 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f2c34-65b3-4dcd-9b88-13a6739dfb01 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:38.823214 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f2c34-65b3-4dcd-9b88-13a6739dfb01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:38.824662 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f2c34-65b3-4dcd-9b88-13a6739dfb01 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:38.845380 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 270/558] 10.222.0.40 () {56 vars in 1135 bytes} [Wed Aug 3 04:54:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:38.845895 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f2c34-65b3-4dcd-9b88-13a6739dfb01 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:38.886150 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b69a9-539d-4df1-8893-7540965c625d None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:38.922992 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ae943-7ed4-4b64-b996-d7c2858ddfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:38.923988 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ae943-7ed4-4b64-b996-d7c2858ddfd5 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:38.924718 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ae943-7ed4-4b64-b996-d7c2858ddfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:38.925326 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ae943-7ed4-4b64-b996-d7c2858ddfd5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:38.936146 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ae943-7ed4-4b64-b996-d7c2858ddfd5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:38.938444 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ae943-7ed4-4b64-b996-d7c2858ddfd5 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:38.953794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-a77ae943-7ed4-4b64-b996-d7c2858ddfd5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 04:54:38.953794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:38.953794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:38.953794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:38.953794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:38.953794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:38.953794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:38.953794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:38.953794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:38.953794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:38.953794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:38.953794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:38.953794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:54:38.953794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:54:38.953794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:54:38.953794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:54:38.954933 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:38.954933 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:38.954933 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:54:38.954933 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:38.954933 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:54:38.954933 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:38.954933 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:38.954933 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:38.954933 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:38.954933 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:38.954933 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:38.954933 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:38.954933 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:38.954933 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:38.954933 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:38.954933 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:38.954933 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:38.954933 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:38.955873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:38.955873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:38.955873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:54:38.955873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:54:38.955873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:54:38.955873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:54:38.955873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:54:38.955873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:54:38.955873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 04:54:38.955873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:38.955873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 289/559] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:54:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:38.956489 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ae943-7ed4-4b64-b996-d7c2858ddfd5 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:38.974630 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f2c34-65b3-4dcd-9b88-13a6739dfb01 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:39.023558 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09821e2c-a7ca-4040-887c-f63b0e1a9ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:39.024993 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09821e2c-a7ca-4040-887c-f63b0e1a9ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:39.025429 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09821e2c-a7ca-4040-887c-f63b0e1a9ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:39.026212 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09821e2c-a7ca-4040-887c-f63b0e1a9ae8 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:39.027527 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09821e2c-a7ca-4040-887c-f63b0e1a9ae8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:39.028249 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09821e2c-a7ca-4040-887c-f63b0e1a9ae8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:39.029791 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09821e2c-a7ca-4040-887c-f63b0e1a9ae8 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:39.045472 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 271/560] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 3 04:54:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:39.047292 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09821e2c-a7ca-4040-887c-f63b0e1a9ae8 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:39.051698 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:39.056929 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:39.082434 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ae943-7ed4-4b64-b996-d7c2858ddfd5 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:39.129664 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef85552-62f9-4096-b309-456febe17a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:39.130574 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef85552-62f9-4096-b309-456febe17a63 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:39.131295 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef85552-62f9-4096-b309-456febe17a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:39.131725 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef85552-62f9-4096-b309-456febe17a63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:39.166492 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09821e2c-a7ca-4040-887c-f63b0e1a9ae8 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:39.212159 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7244c08-6683-42b1-8262-b9721245085b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:39.213620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7244c08-6683-42b1-8262-b9721245085b None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:39.214056 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7244c08-6683-42b1-8262-b9721245085b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:39.214593 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7244c08-6683-42b1-8262-b9721245085b None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:39.215674 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef85552-62f9-4096-b309-456febe17a63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:39.215969 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7244c08-6683-42b1-8262-b9721245085b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:39.216422 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7244c08-6683-42b1-8262-b9721245085b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:39.217442 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef85552-62f9-4096-b309-456febe17a63 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:39.218236 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7244c08-6683-42b1-8262-b9721245085b None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:39.245717 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 272/561] 10.222.0.40 () {56 vars in 1137 bytes} [Wed Aug 3 04:54:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:39.247532 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7244c08-6683-42b1-8262-b9721245085b None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:39.311237 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:39.320247 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:39.330832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 290/562] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:54:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:54:39.331537 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef85552-62f9-4096-b309-456febe17a63 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:39.369519 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7244c08-6683-42b1-8262-b9721245085b None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:39.407611 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43106a-e45e-48e3-92f2-75095e51b1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:39.408250 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43106a-e45e-48e3-92f2-75095e51b1cb None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:39.408530 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43106a-e45e-48e3-92f2-75095e51b1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:39.408803 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43106a-e45e-48e3-92f2-75095e51b1cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:39.419301 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d43106a-e45e-48e3-92f2-75095e51b1cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:39.420769 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d43106a-e45e-48e3-92f2-75095e51b1cb None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:39.436324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-1d43106a-e45e-48e3-92f2-75095e51b1cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:39.436324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:39.436324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:39.436324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:39.436324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:39.436324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:39.436324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:39.436324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:39.436324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:39.436324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:39.436324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:39.436324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:39.436324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:39.436324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:39.436324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:39.437298 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:39.437298 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:39.437298 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:39.437298 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:39.437298 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:39.437298 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:39.437298 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:39.437298 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:39.437298 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:39.437298 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:39.437298 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:39.437298 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:39.437298 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:39.437298 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:39.437298 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:39.437298 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:39.437298 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:39.438186 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:39.438186 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:39.438186 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:39.438186 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:39.438186 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 273/563] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:39.439141 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba593e-ab39-4d6c-bcd7-7e02de7ddf6d None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:39.439407 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba593e-ab39-4d6c-bcd7-7e02de7ddf6d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:39.439679 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba593e-ab39-4d6c-bcd7-7e02de7ddf6d None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:39.441039 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 274/564] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:39.445452 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba593e-ab39-4d6c-bcd7-7e02de7ddf6d None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:39.449831 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef85552-62f9-4096-b309-456febe17a63 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:39.487989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75336ca-8394-4870-a4d5-1e693ffb5c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:39.488635 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75336ca-8394-4870-a4d5-1e693ffb5c7e None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:39.488922 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75336ca-8394-4870-a4d5-1e693ffb5c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:39.489204 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75336ca-8394-4870-a4d5-1e693ffb5c7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:39.499168 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75336ca-8394-4870-a4d5-1e693ffb5c7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:39.500691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75336ca-8394-4870-a4d5-1e693ffb5c7e None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:39.514894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-a75336ca-8394-4870-a4d5-1e693ffb5c7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:39.514894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:39.514894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:39.514894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:39.514894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:39.514894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:39.514894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:39.514894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:39.514894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:39.514894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:39.514894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:39.514894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:39.514894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:39.514894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:39.514894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:39.516470 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:39.516470 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:39.516470 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:39.516470 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:39.516470 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:39.516470 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:39.516470 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:39.516470 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:39.516470 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:39.516470 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:39.516470 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:39.516470 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:39.516470 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:39.516470 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:39.516470 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:39.516470 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:39.516470 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:39.517579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:39.517579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:39.517579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:39.517579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:39.517579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 291/565] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:39.517579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75336ca-8394-4870-a4d5-1e693ffb5c7e None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:39.551379 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:39.553623 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba593e-ab39-4d6c-bcd7-7e02de7ddf6d None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:39.567899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:39.587854 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd24c950-d8b4-42f5-a175-26dafacf10d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:39.588490 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24c950-d8b4-42f5-a175-26dafacf10d4 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:39.588773 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24c950-d8b4-42f5-a175-26dafacf10d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:39.589049 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24c950-d8b4-42f5-a175-26dafacf10d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:39.600206 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd24c950-d8b4-42f5-a175-26dafacf10d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:39.601670 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd24c950-d8b4-42f5-a175-26dafacf10d4 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:39.615701 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-dd24c950-d8b4-42f5-a175-26dafacf10d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:39.615701 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:39.615701 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:39.615701 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:39.615701 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:39.615701 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:39.615701 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:39.615701 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:39.615701 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:39.615701 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:39.615701 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:39.615701 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:39.615701 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:39.615701 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:39.615701 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:39.616601 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:39.616601 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:39.616601 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:39.616601 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:39.616601 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:39.616601 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:39.616601 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:39.616601 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:39.616601 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:39.616601 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:39.616601 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:39.616601 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:39.616601 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:39.616601 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:39.616601 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:39.616601 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:39.616601 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:39.617512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:39.617512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:39.617512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:39.617512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:39.617793 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75336ca-8394-4870-a4d5-1e693ffb5c7e None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:39.621500 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 275/566] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:39.622715 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4a913-c97f-4325-8481-1592e22379fd None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:39.623035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4a913-c97f-4325-8481-1592e22379fd None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:39.623320 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4a913-c97f-4325-8481-1592e22379fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:39.642706 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-c0c4a913-c97f-4325-8481-1592e22379fd None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:39.643949 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c0c4a913-c97f-4325-8481-1592e22379fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:39.652774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b421a9-9cdf-42ae-a7c5-3e656c25f0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:39.653390 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b421a9-9cdf-42ae-a7c5-3e656c25f0f5 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:39.653670 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b421a9-9cdf-42ae-a7c5-3e656c25f0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:39.654006 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b421a9-9cdf-42ae-a7c5-3e656c25f0f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:39.654752 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b421a9-9cdf-42ae-a7c5-3e656c25f0f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:39.655077 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b421a9-9cdf-42ae-a7c5-3e656c25f0f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:39.656464 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b421a9-9cdf-42ae-a7c5-3e656c25f0f5 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:39.668071 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 292/567] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:39.669461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827d9ab-78f4-403d-9441-33a8d605fee2 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:39.669737 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827d9ab-78f4-403d-9441-33a8d605fee2 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:39.670010 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827d9ab-78f4-403d-9441-33a8d605fee2 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:39.675147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 293/568] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:39.675147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827d9ab-78f4-403d-9441-33a8d605fee2 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:39.719742 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 276/569] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:39.720857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4a913-c97f-4325-8481-1592e22379fd None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:39.779877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:39.787962 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827d9ab-78f4-403d-9441-33a8d605fee2 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:39.815311 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:39.826329 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4a913-c97f-4325-8481-1592e22379fd None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:39.826492 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09533982-d65a-4f33-bc70-077a97ffe115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:39.826492 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09533982-d65a-4f33-bc70-077a97ffe115 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:39.826492 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09533982-d65a-4f33-bc70-077a97ffe115 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:39.826492 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09533982-d65a-4f33-bc70-077a97ffe115 None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:39.826863 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09533982-d65a-4f33-bc70-077a97ffe115 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:39.826863 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09533982-d65a-4f33-bc70-077a97ffe115 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:39.826863 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09533982-d65a-4f33-bc70-077a97ffe115 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:39.838145 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 294/570] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:39.838892 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09533982-d65a-4f33-bc70-077a97ffe115 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:39.855598 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cbaee4-6285-4c68-a7b2-0bc84233073a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:39.856219 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cbaee4-6285-4c68-a7b2-0bc84233073a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:39.856508 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cbaee4-6285-4c68-a7b2-0bc84233073a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:39.856814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cbaee4-6285-4c68-a7b2-0bc84233073a None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:39.857484 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cbaee4-6285-4c68-a7b2-0bc84233073a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:39.857793 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cbaee4-6285-4c68-a7b2-0bc84233073a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:39.861061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cbaee4-6285-4c68-a7b2-0bc84233073a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:39.876050 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 277/571] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:39.877604 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a516e2-6142-4142-bb80-dd731a8dc6fe None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:39.878080 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a516e2-6142-4142-bb80-dd731a8dc6fe None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:39.878369 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a516e2-6142-4142-bb80-dd731a8dc6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:39.896321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-a7a516e2-6142-4142-bb80-dd731a8dc6fe None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:39.897527 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a7a516e2-6142-4142-bb80-dd731a8dc6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:39.959256 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09533982-d65a-4f33-bc70-077a97ffe115 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:39.964561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 278/572] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:39.965394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a516e2-6142-4142-bb80-dd731a8dc6fe None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:40.000191 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8d3b6-e22b-42a5-bcdf-ca4042b72f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:40.000910 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8d3b6-e22b-42a5-bcdf-ca4042b72f7b None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:40.001192 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8d3b6-e22b-42a5-bcdf-ca4042b72f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:40.001466 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8d3b6-e22b-42a5-bcdf-ca4042b72f7b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:40.009109 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e8d3b6-e22b-42a5-bcdf-ca4042b72f7b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:40.010622 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e8d3b6-e22b-42a5-bcdf-ca4042b72f7b None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:40.022111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-45e8d3b6-e22b-42a5-bcdf-ca4042b72f7b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 04:54:40.022111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:40.022111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:40.022111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:40.022111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:40.022111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:40.022111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:40.022111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:40.022111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:40.022111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:40.022111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:40.022111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:40.022111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:54:40.022111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:54:40.022111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:54:40.022111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:54:40.023226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:40.023226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:40.023226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:54:40.023226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:40.023226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:54:40.023226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:40.023226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:40.023226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:40.023226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:40.023226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:40.023226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:40.023226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:40.023226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:40.023226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:40.023226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:40.023226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:40.023226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:40.023226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:40.024979 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:40.024979 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:40.024979 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:54:40.024979 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:54:40.024979 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:54:40.024979 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:54:40.024979 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:54:40.024979 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:54:40.024979 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 04:54:40.024979 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:40.024979 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 295/573] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Aug 3 04:54:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:40.024979 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8d3b6-e22b-42a5-bcdf-ca4042b72f7b None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:40.039512 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:40.058572 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:40.073455 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a516e2-6142-4142-bb80-dd731a8dc6fe None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:40.107970 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2529e7-19d2-41e6-842a-ff2f2c70bc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:40.108528 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2529e7-19d2-41e6-842a-ff2f2c70bc67 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:40.109373 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2529e7-19d2-41e6-842a-ff2f2c70bc67 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:40.109646 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2529e7-19d2-41e6-842a-ff2f2c70bc67 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:40.120206 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2529e7-19d2-41e6-842a-ff2f2c70bc67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:40.125636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-6f2529e7-19d2-41e6-842a-ff2f2c70bc67 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:40.125636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:40.125636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:40.125636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:40.125636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:40.125636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:40.125636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:40.125636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:40.125636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:40.125636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:40.125636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:40.125636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:40.125636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:40.125636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:40.125636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:40.125636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:40.126617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:40.126617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:40.126617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:40.126617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:40.126617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:40.126617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:40.126617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:40.126617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:40.126617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:40.126617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:40.126617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:40.126617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:40.126617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:40.126617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:40.126617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:40.126617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:40.127646 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:40.127646 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:40.127646 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:40.127646 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:40.127646 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:40.127646 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:40.127646 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:40.127646 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:40.127646 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:40.127646 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:40.127646 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:40.127646 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:40.128279 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8d3b6-e22b-42a5-bcdf-ca4042b72f7b None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:40.128354 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 279/574] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 3 04:54:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:40.128354 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2529e7-19d2-41e6-842a-ff2f2c70bc67 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:40.164527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7ab0d-e170-41c5-b7d8-cc0a206a2b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:40.165160 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7ab0d-e170-41c5-b7d8-cc0a206a2b11 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:40.165441 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7ab0d-e170-41c5-b7d8-cc0a206a2b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:40.165712 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7ab0d-e170-41c5-b7d8-cc0a206a2b11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:40.172971 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b7ab0d-e170-41c5-b7d8-cc0a206a2b11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:40.174443 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b7ab0d-e170-41c5-b7d8-cc0a206a2b11 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:40.185706 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-e3b7ab0d-e170-41c5-b7d8-cc0a206a2b11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:40.185706 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:40.185706 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:40.185706 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:40.185706 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:40.185706 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:40.185706 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:40.185706 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:40.185706 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:40.185706 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:40.185706 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:40.185706 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:40.185706 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:54:40.185706 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:54:40.185706 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:54:40.186609 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:54:40.186609 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:40.186609 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:40.186609 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:40.186609 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:40.186609 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:40.186609 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:40.186609 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:40.186609 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:40.186609 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:40.186609 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:40.186609 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:40.186609 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:40.186609 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:40.186609 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:40.186609 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:40.186609 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:40.188395 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:54:40.188395 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:54:40.188395 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:54:40.188395 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:54:40.188395 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:54:40.188395 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:54:40.188395 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:40.188395 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:40.188395 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 296/575] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:54:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:40.188395 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7ab0d-e170-41c5-b7d8-cc0a206a2b11 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:40.233315 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2529e7-19d2-41e6-842a-ff2f2c70bc67 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:40.263856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:40.268271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95613ae-7391-4830-a30a-6115163e4e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:40.268965 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95613ae-7391-4830-a30a-6115163e4e34 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:40.271953 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95613ae-7391-4830-a30a-6115163e4e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:40.271953 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95613ae-7391-4830-a30a-6115163e4e34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:40.276655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95613ae-7391-4830-a30a-6115163e4e34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:40.278226 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95613ae-7391-4830-a30a-6115163e4e34 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:40.289367 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7ab0d-e170-41c5-b7d8-cc0a206a2b11 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:40.289939 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-f95613ae-7391-4830-a30a-6115163e4e34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:40.289939 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:40.289939 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:40.289939 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:40.289939 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:40.289939 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:40.289939 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:40.289939 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:40.289939 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:40.289939 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:40.289939 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:40.289939 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:40.289939 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:54:40.289939 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:54:40.289939 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:54:40.290899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:54:40.290899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:40.290899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:40.290899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:40.290899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:40.290899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:40.290899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:40.290899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:40.290899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:40.290899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:40.290899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:40.290899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:40.290899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:40.290899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:40.290899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:40.290899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:40.290899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:40.291793 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:54:40.291793 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:54:40.291793 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:54:40.291793 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:54:40.291793 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:54:40.291793 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:54:40.291793 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:40.291793 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:40.291793 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 280/576] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:54:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:40.292787 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95613ae-7391-4830-a30a-6115163e4e34 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:40.302605 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:40.328398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5e694-f46b-47c9-b314-c28a34c3de4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:40.329037 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5e694-f46b-47c9-b314-c28a34c3de4c None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:40.329371 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5e694-f46b-47c9-b314-c28a34c3de4c None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:40.329647 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5e694-f46b-47c9-b314-c28a34c3de4c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:40.340124 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae5e694-f46b-47c9-b314-c28a34c3de4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:40.345879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-2ae5e694-f46b-47c9-b314-c28a34c3de4c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:40.345879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:40.345879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:40.345879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:40.345879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:40.345879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:40.345879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:40.345879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:40.345879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:40.345879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:40.345879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:40.345879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:40.345879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:40.345879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:40.345879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:40.345879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:40.346805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:40.346805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:40.346805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:40.346805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:40.346805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:40.346805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:40.346805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:40.346805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:40.346805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:40.346805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:40.346805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:40.346805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:40.346805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:40.346805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:40.346805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:40.346805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:40.352903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:40.352903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:40.352903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:40.352903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:40.352903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:40.352903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:40.352903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:40.352903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:40.352903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:40.352903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:40.352903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:40.352903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:40.352903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 297/577] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 3 04:54:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:40.352903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5e694-f46b-47c9-b314-c28a34c3de4c None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:40.401883 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95613ae-7391-4830-a30a-6115163e4e34 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:40.437044 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d88507-5ed2-4202-9be8-6b944f07ad17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:40.437655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d88507-5ed2-4202-9be8-6b944f07ad17 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:40.437940 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d88507-5ed2-4202-9be8-6b944f07ad17 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:40.438559 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d88507-5ed2-4202-9be8-6b944f07ad17 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:40.439036 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d88507-5ed2-4202-9be8-6b944f07ad17 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:40.439350 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d88507-5ed2-4202-9be8-6b944f07ad17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:40.440791 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d88507-5ed2-4202-9be8-6b944f07ad17 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:40.450570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5e694-f46b-47c9-b314-c28a34c3de4c None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:40.458164 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 281/578] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Aug 3 04:54:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:40.458997 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d88507-5ed2-4202-9be8-6b944f07ad17 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:40.489017 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d5674-9772-4360-baa1-7636fdfc4c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:40.489634 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d5674-9772-4360-baa1-7636fdfc4c77 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:40.489919 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d5674-9772-4360-baa1-7636fdfc4c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:40.490201 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d5674-9772-4360-baa1-7636fdfc4c77 None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:40.490951 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d5674-9772-4360-baa1-7636fdfc4c77 None admin] RBAC: Adding query filter params (name=service) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:40.491262 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d5674-9772-4360-baa1-7636fdfc4c77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:40.492657 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d5674-9772-4360-baa1-7636fdfc4c77 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:40.493994 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:40.501396 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 298/579] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 3 04:54:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:40.502193 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d5674-9772-4360-baa1-7636fdfc4c77 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:40.527790 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:40.566109 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d88507-5ed2-4202-9be8-6b944f07ad17 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:40.599971 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621d9cf-9e91-46fc-8d56-a9d92695fb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:40.600852 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621d9cf-9e91-46fc-8d56-a9d92695fb05 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:40.600852 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621d9cf-9e91-46fc-8d56-a9d92695fb05 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:40.601132 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621d9cf-9e91-46fc-8d56-a9d92695fb05 None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:40.601852 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621d9cf-9e91-46fc-8d56-a9d92695fb05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:40.602161 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621d9cf-9e91-46fc-8d56-a9d92695fb05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:40.603423 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d5674-9772-4360-baa1-7636fdfc4c77 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:40.603622 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621d9cf-9e91-46fc-8d56-a9d92695fb05 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:40.615624 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 282/580] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:54:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:40.616663 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621d9cf-9e91-46fc-8d56-a9d92695fb05 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:40.642923 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95295c-20aa-41f8-8c4e-a8e7b79fecd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:40.643548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95295c-20aa-41f8-8c4e-a8e7b79fecd1 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:40.643830 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95295c-20aa-41f8-8c4e-a8e7b79fecd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:40.644108 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95295c-20aa-41f8-8c4e-a8e7b79fecd1 None admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:40.644852 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95295c-20aa-41f8-8c4e-a8e7b79fecd1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:40.645158 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95295c-20aa-41f8-8c4e-a8e7b79fecd1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:40.646620 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95295c-20aa-41f8-8c4e-a8e7b79fecd1 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:40.661460 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 299/581] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:54:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:40.661460 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95295c-20aa-41f8-8c4e-a8e7b79fecd1 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:40.721974 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:40.723357 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621d9cf-9e91-46fc-8d56-a9d92695fb05 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:40.746064 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:40.756337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bea7d9-b09a-4995-b728-95f033ecf78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:40.756849 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bea7d9-b09a-4995-b728-95f033ecf78c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:40.757134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bea7d9-b09a-4995-b728-95f033ecf78c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:40.757410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bea7d9-b09a-4995-b728-95f033ecf78c None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:40.758124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bea7d9-b09a-4995-b728-95f033ecf78c None admin] RBAC: Adding query filter params (name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:40.758432 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bea7d9-b09a-4995-b728-95f033ecf78c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:40.759844 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bea7d9-b09a-4995-b728-95f033ecf78c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:40.767306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95295c-20aa-41f8-8c4e-a8e7b79fecd1 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:40.768790 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 283/582] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 3 04:54:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:40.769613 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bea7d9-b09a-4995-b728-95f033ecf78c None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:40.805785 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95c06c1-fa98-44f8-ad38-9c67859180ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:40.806426 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c06c1-fa98-44f8-ad38-9c67859180ed None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:40.806710 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c06c1-fa98-44f8-ad38-9c67859180ed None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:40.807018 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c06c1-fa98-44f8-ad38-9c67859180ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:40.817504 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95c06c1-fa98-44f8-ad38-9c67859180ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:40.818984 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95c06c1-fa98-44f8-ad38-9c67859180ed None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:40.834610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-f95c06c1-fa98-44f8-ad38-9c67859180ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:40.834610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:40.834610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:40.834610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:40.834610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:40.834610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:40.834610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:40.834610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:40.834610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:40.834610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:40.834610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:40.834610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:40.834610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:40.834610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:40.834610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:40.835541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:40.835541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:40.835541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:40.835541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:40.835541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:40.835541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:40.835541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:40.835541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:40.835541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:40.835541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:40.835541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:40.835541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:40.835541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:40.835541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:40.835541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:40.835541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:40.835541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:40.836426 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:40.836426 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:40.836426 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:40.836426 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:40.836426 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 300/583] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:40.836698 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95c06c1-fa98-44f8-ad38-9c67859180ed None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:40.871719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bea7d9-b09a-4995-b728-95f033ecf78c None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:40.905469 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df761c5-01fb-4d7b-a8e2-86b09b1665ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:40.906220 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df761c5-01fb-4d7b-a8e2-86b09b1665ce None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:40.906503 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df761c5-01fb-4d7b-a8e2-86b09b1665ce None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:40.906777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df761c5-01fb-4d7b-a8e2-86b09b1665ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:40.916764 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df761c5-01fb-4d7b-a8e2-86b09b1665ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:40.918149 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df761c5-01fb-4d7b-a8e2-86b09b1665ce None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:40.933260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-4df761c5-01fb-4d7b-a8e2-86b09b1665ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:40.933260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:40.933260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:40.933260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:40.933260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:40.933260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:40.933260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:40.933260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:40.933260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:40.933260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:40.933260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:40.933260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:40.933260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:40.933260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:40.933260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:40.934337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:40.934337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:40.934337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:40.934337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:40.934337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:40.934337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:40.934337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:40.934337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:40.934337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:40.934337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:40.934337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:40.934337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:40.934337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:40.934337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:40.934337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:40.934337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:40.934337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:40.935491 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:40.935491 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:40.935491 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:40.935491 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:40.937552 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 284/584] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:40.938477 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df761c5-01fb-4d7b-a8e2-86b09b1665ce None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:40.943935 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95c06c1-fa98-44f8-ad38-9c67859180ed None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:40.952960 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:40.978238 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:40.978874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490f81a-6f85-4310-9f40-30173b32c707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:40.979484 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490f81a-6f85-4310-9f40-30173b32c707 None admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:40.979765 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490f81a-6f85-4310-9f40-30173b32c707 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:40.980041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490f81a-6f85-4310-9f40-30173b32c707 None admin] PATH_INFO: `/v3/projects/cf3763a37383456184a9ca9351b4ce03/users/c81ca55d52244bf1bfe4e829b6880c20/roles/cae571342db14f5fac7849efe7f8636e` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:40.997135 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5490f81a-6f85-4310-9f40-30173b32c707 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf3763a37383456184a9ca9351b4ce03, user_id=c81ca55d52244bf1bfe4e829b6880c20, role_id=cae571342db14f5fac7849efe7f8636e)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:41.013679 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5490f81a-6f85-4310-9f40-30173b32c707 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:41.038573 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df761c5-01fb-4d7b-a8e2-86b09b1665ce None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:41.072655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00103725-3b17-4d81-96ca-bbe2ca59a611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:41.073273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00103725-3b17-4d81-96ca-bbe2ca59a611 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:41.073559 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00103725-3b17-4d81-96ca-bbe2ca59a611 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:41.073839 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00103725-3b17-4d81-96ca-bbe2ca59a611 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:41.074617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00103725-3b17-4d81-96ca-bbe2ca59a611 None admin] RBAC: Adding query filter params (role.id=cae571342db14f5fac7849efe7f8636e, scope.project.id=cf3763a37383456184a9ca9351b4ce03, user.id=e1014bcb4d36441aa46042941c9a63d0) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:41.074957 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00103725-3b17-4d81-96ca-bbe2ca59a611 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:41.076384 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00103725-3b17-4d81-96ca-bbe2ca59a611 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:41.081844 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 285/585] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:54:40 2022] GET /identity/v3/role_assignments?user.id=e1014bcb4d36441aa46042941c9a63d0&scope.project.id=cf3763a37383456184a9ca9351b4ce03&role.id=cae571342db14f5fac7849efe7f8636e => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:41.082614 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00103725-3b17-4d81-96ca-bbe2ca59a611 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:41.122873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 301/586] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 3 04:54:40 2022] PUT /identity/v3/projects/cf3763a37383456184a9ca9351b4ce03/users/c81ca55d52244bf1bfe4e829b6880c20/roles/cae571342db14f5fac7849efe7f8636e => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 04:54:41.122873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490f81a-6f85-4310-9f40-30173b32c707 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:41.225951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00103725-3b17-4d81-96ca-bbe2ca59a611 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:41.228223 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:41.264323 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490f81a-6f85-4310-9f40-30173b32c707 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:41.271021 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1b485-f224-4991-8e11-5eddfcc373cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:41.271794 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1b485-f224-4991-8e11-5eddfcc373cd None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:41.272157 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1b485-f224-4991-8e11-5eddfcc373cd None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:41.272513 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1b485-f224-4991-8e11-5eddfcc373cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:41.295252 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee1b485-f224-4991-8e11-5eddfcc373cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:41.297791 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee1b485-f224-4991-8e11-5eddfcc373cd None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:41.312405 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ff7a6-70af-4360-bf0f-87925af68c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:41.313328 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ff7a6-70af-4360-bf0f-87925af68c72 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:41.313785 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ff7a6-70af-4360-bf0f-87925af68c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:41.314270 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ff7a6-70af-4360-bf0f-87925af68c72 None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:41.315004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ff7a6-70af-4360-bf0f-87925af68c72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:41.315307 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ff7a6-70af-4360-bf0f-87925af68c72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:41.316882 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ff7a6-70af-4360-bf0f-87925af68c72 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:41.319184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-4ee1b485-f224-4991-8e11-5eddfcc373cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:41.319184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:41.319184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:41.319184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:41.319184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:41.319184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:41.319184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:41.319184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:41.319184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:41.319184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:41.319184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:41.319184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:41.319184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:41.319184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:41.319184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:41.320076 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:41.320076 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:41.320076 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:41.320076 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:41.320076 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:41.320076 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:41.320076 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:41.320076 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:41.320076 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:41.320076 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:41.320076 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:41.320076 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:41.320076 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:41.320076 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:41.320076 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:41.320076 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:41.320076 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:41.320977 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:41.320977 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:41.320977 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:41.320977 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:41.320977 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 286/587] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:41.321266 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1b485-f224-4991-8e11-5eddfcc373cd None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:41.333106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 302/588] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:41.334482 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ff7a6-70af-4360-bf0f-87925af68c72 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:41.372439 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:41.473645 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1b485-f224-4991-8e11-5eddfcc373cd None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:41.498400 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ff7a6-70af-4360-bf0f-87925af68c72 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:41.509781 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8bd02a-6520-4eb9-ab98-fbe48a0c7dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:41.510692 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bd02a-6520-4eb9-ab98-fbe48a0c7dbc None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:41.511078 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bd02a-6520-4eb9-ab98-fbe48a0c7dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:41.511503 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bd02a-6520-4eb9-ab98-fbe48a0c7dbc None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:41.512266 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8bd02a-6520-4eb9-ab98-fbe48a0c7dbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:41.512638 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8bd02a-6520-4eb9-ab98-fbe48a0c7dbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:41.514175 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8bd02a-6520-4eb9-ab98-fbe48a0c7dbc None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:41.530212 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 287/589] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:41.531903 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8bd02a-6520-4eb9-ab98-fbe48a0c7dbc None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:41.542779 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09090811-33a1-4cf7-91b5-7f9769f8c608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:41.543433 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09090811-33a1-4cf7-91b5-7f9769f8c608 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:41.543786 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09090811-33a1-4cf7-91b5-7f9769f8c608 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:41.544134 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09090811-33a1-4cf7-91b5-7f9769f8c608 None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:41.544826 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09090811-33a1-4cf7-91b5-7f9769f8c608 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:41.545127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09090811-33a1-4cf7-91b5-7f9769f8c608 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:41.546722 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09090811-33a1-4cf7-91b5-7f9769f8c608 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:41.549207 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:41.565699 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 303/590] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:41.566797 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09090811-33a1-4cf7-91b5-7f9769f8c608 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:41.661113 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8bd02a-6520-4eb9-ab98-fbe48a0c7dbc None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:41.670110 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:41.693962 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09090811-33a1-4cf7-91b5-7f9769f8c608 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:41.707317 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5850cf2-642a-482b-87ae-978b77ef6bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:41.709230 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5850cf2-642a-482b-87ae-978b77ef6bcf None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:41.710334 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5850cf2-642a-482b-87ae-978b77ef6bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:41.710758 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5850cf2-642a-482b-87ae-978b77ef6bcf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:41.721253 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5850cf2-642a-482b-87ae-978b77ef6bcf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:41.723063 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5850cf2-642a-482b-87ae-978b77ef6bcf None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:41.734823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-f5850cf2-642a-482b-87ae-978b77ef6bcf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:41.734823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:41.734823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:41.734823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:41.734823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:41.734823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:41.734823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:41.734823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:41.734823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:41.734823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:41.734823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:41.734823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:41.734823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:54:41.734823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:54:41.734823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:54:41.735836 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:54:41.735836 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:41.735836 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:41.735836 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:41.735836 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:41.735836 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:41.735836 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:41.735836 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:41.735836 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:41.735836 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:41.735836 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:41.735836 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:41.735836 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:41.735836 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:41.735836 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:41.735836 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:41.735836 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:41.736722 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:54:41.736722 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:54:41.736722 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:54:41.736722 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:54:41.736722 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:54:41.736722 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:54:41.736722 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:41.736722 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:41.739161 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 288/591] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:54:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:41.740137 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5850cf2-642a-482b-87ae-978b77ef6bcf None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:41.743358 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f1668-6fb6-49af-b6b7-8b56ccf10e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:41.744266 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f1668-6fb6-49af-b6b7-8b56ccf10e8c None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:41.744743 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f1668-6fb6-49af-b6b7-8b56ccf10e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:41.745193 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f1668-6fb6-49af-b6b7-8b56ccf10e8c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:41.758220 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f1668-6fb6-49af-b6b7-8b56ccf10e8c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:41.760018 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f1668-6fb6-49af-b6b7-8b56ccf10e8c None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:41.773165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-810f1668-6fb6-49af-b6b7-8b56ccf10e8c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 03 04:54:41.773165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:41.773165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:41.773165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:41.773165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:41.773165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:41.773165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:41.773165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:41.773165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:41.773165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:41.773165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:41.773165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:41.773165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:54:41.773165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:54:41.773165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:54:41.773165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:54:41.774420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:41.774420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:41.774420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:54:41.774420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:41.774420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:54:41.774420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:41.774420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:41.774420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:41.774420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:41.774420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:41.774420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:41.774420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:41.774420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:41.774420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:41.774420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:41.774420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:41.774420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:41.774420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:41.775386 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:41.775386 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:41.775386 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:54:41.775386 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:54:41.775386 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:54:41.775386 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:54:41.775386 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:54:41.775386 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:54:41.775386 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 03 04:54:41.775386 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:41.783058 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 304/592] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:54:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:41.784103 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f1668-6fb6-49af-b6b7-8b56ccf10e8c None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:41.849499 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:41.880161 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5850cf2-642a-482b-87ae-978b77ef6bcf None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:41.923353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f1668-6fb6-49af-b6b7-8b56ccf10e8c None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:41.927251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c6a87-2945-473b-a66d-f04716a0501e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:41.927921 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c6a87-2945-473b-a66d-f04716a0501e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:41.928203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c6a87-2945-473b-a66d-f04716a0501e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:41.928479 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c6a87-2945-473b-a66d-f04716a0501e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:41.940380 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0c6a87-2945-473b-a66d-f04716a0501e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:41.942647 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0c6a87-2945-473b-a66d-f04716a0501e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:41.963545 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-ba0c6a87-2945-473b-a66d-f04716a0501e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 03 04:54:41.963545 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:41.963545 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:41.963545 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:41.963545 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:41.963545 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:41.963545 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:41.963545 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:41.963545 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:41.963545 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:41.963545 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:41.963545 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:41.963545 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:54:41.963545 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:54:41.963545 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:54:41.963545 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:54:41.964548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:41.964548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:41.964548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:54:41.964548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:41.964548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:54:41.964548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:41.964548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:41.964548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:41.964548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:41.964548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:41.964548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:41.964548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:41.964548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:41.964548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:41.964548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:41.964548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:41.964548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:41.964548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:41.965489 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:41.965489 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:41.965489 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:54:41.965489 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:54:41.965489 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:54:41.965489 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:54:41.965489 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:54:41.965489 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:54:41.965489 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 03 04:54:41.965489 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:41.978616 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 289/593] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 3 04:54:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:41.979916 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c6a87-2945-473b-a66d-f04716a0501e None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:41.986455 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e9a99-0c8c-492c-acc5-78bbb3a00189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:41.987518 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e9a99-0c8c-492c-acc5-78bbb3a00189 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:41.988001 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e9a99-0c8c-492c-acc5-78bbb3a00189 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:41.988468 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e9a99-0c8c-492c-acc5-78bbb3a00189 None admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:41.989440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e9a99-0c8c-492c-acc5-78bbb3a00189 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:41.989950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e9a99-0c8c-492c-acc5-78bbb3a00189 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:41.991673 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e9a99-0c8c-492c-acc5-78bbb3a00189 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:41.999910 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:42.007054 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 305/594] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:54:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:42.007054 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e9a99-0c8c-492c-acc5-78bbb3a00189 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:42.107254 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c6a87-2945-473b-a66d-f04716a0501e None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:42.108674 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e9a99-0c8c-492c-acc5-78bbb3a00189 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:42.144491 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9951b4-56c2-4c72-b48f-a1e7be73bd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:42.145489 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9951b4-56c2-4c72-b48f-a1e7be73bd9b None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:42.145903 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd0fe6-db06-4bf7-ad4e-d493507561cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:42.146174 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9951b4-56c2-4c72-b48f-a1e7be73bd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:42.146622 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9951b4-56c2-4c72-b48f-a1e7be73bd9b None admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:42.146725 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd0fe6-db06-4bf7-ad4e-d493507561cd None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:42.147039 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd0fe6-db06-4bf7-ad4e-d493507561cd None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:42.147387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd0fe6-db06-4bf7-ad4e-d493507561cd None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:42.148088 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9951b4-56c2-4c72-b48f-a1e7be73bd9b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:42.148237 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddd0fe6-db06-4bf7-ad4e-d493507561cd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:42.148696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddd0fe6-db06-4bf7-ad4e-d493507561cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:42.149102 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9951b4-56c2-4c72-b48f-a1e7be73bd9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:42.150646 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddd0fe6-db06-4bf7-ad4e-d493507561cd None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:42.150968 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9951b4-56c2-4c72-b48f-a1e7be73bd9b None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:42.152439 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:42.170235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 306/595] 10.222.0.40 () {56 vars in 1139 bytes} [Wed Aug 3 04:54:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:42.172202 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9951b4-56c2-4c72-b48f-a1e7be73bd9b None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:42.180796 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 290/596] 10.222.0.40 () {56 vars in 1137 bytes} [Wed Aug 3 04:54:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:42.180796 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd0fe6-db06-4bf7-ad4e-d493507561cd None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:42.235024 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:42.286100 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9951b4-56c2-4c72-b48f-a1e7be73bd9b None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:42.297255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd0fe6-db06-4bf7-ad4e-d493507561cd None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:42.328532 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349f7f3-c65d-4ff4-ba34-1bfae5534981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:42.329396 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349f7f3-c65d-4ff4-ba34-1bfae5534981 None admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:42.329757 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349f7f3-c65d-4ff4-ba34-1bfae5534981 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:42.330265 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349f7f3-c65d-4ff4-ba34-1bfae5534981 None admin] PATH_INFO: `/v3/projects/cf3763a37383456184a9ca9351b4ce03/users/a3fa4a6c16274c18849869b55b33ac62/roles/cae571342db14f5fac7849efe7f8636e` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:42.340366 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a61fb-03b4-4cb3-83fc-3c13b48e3bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:42.341016 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a61fb-03b4-4cb3-83fc-3c13b48e3bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:42.341303 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a61fb-03b4-4cb3-83fc-3c13b48e3bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:42.341579 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a61fb-03b4-4cb3-83fc-3c13b48e3bf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:42.352137 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5a61fb-03b4-4cb3-83fc-3c13b48e3bf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:42.353597 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5a61fb-03b4-4cb3-83fc-3c13b48e3bf6 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:42.368677 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349f7f3-c65d-4ff4-ba34-1bfae5534981 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf3763a37383456184a9ca9351b4ce03, user_id=a3fa4a6c16274c18849869b55b33ac62, role_id=cae571342db14f5fac7849efe7f8636e)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:42.370144 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349f7f3-c65d-4ff4-ba34-1bfae5534981 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:42.372073 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-7d5a61fb-03b4-4cb3-83fc-3c13b48e3bf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:42.372073 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:42.372073 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:42.372073 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:42.372073 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:42.372073 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:42.372073 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:42.372073 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:42.372073 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:42.372073 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:42.372073 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:42.372073 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:42.372073 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:42.372073 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:42.372073 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:42.372923 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:42.372923 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:42.372923 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:42.372923 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:42.372923 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:42.372923 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:42.372923 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:42.372923 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:42.372923 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:42.372923 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:42.372923 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:42.372923 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:42.372923 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:42.372923 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:42.372923 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:42.372923 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:42.372923 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:42.373806 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:42.373806 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:42.373806 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:42.373806 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:42.377091 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 291/597] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:54:42.378177 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a61fb-03b4-4cb3-83fc-3c13b48e3bf6 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:42.427357 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:42.471239 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 307/598] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 3 04:54:42 2022] PUT /identity/v3/projects/cf3763a37383456184a9ca9351b4ce03/users/a3fa4a6c16274c18849869b55b33ac62/roles/cae571342db14f5fac7849efe7f8636e => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 04:54:42.473083 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349f7f3-c65d-4ff4-ba34-1bfae5534981 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:42.502809 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a61fb-03b4-4cb3-83fc-3c13b48e3bf6 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:42.577685 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09650b16-e899-482e-8313-be6eb1273a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:42.578505 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09650b16-e899-482e-8313-be6eb1273a41 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:42.578872 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09650b16-e899-482e-8313-be6eb1273a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:42.579160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09650b16-e899-482e-8313-be6eb1273a41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:42.605826 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09650b16-e899-482e-8313-be6eb1273a41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:42.607343 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09650b16-e899-482e-8313-be6eb1273a41 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:42.636985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-09650b16-e899-482e-8313-be6eb1273a41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:42.636985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:42.636985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:42.636985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:42.636985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:42.636985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:42.636985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:42.636985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:42.636985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:42.636985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:42.636985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:42.636985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:42.636985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:42.636985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:42.636985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:42.637908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:42.637908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:42.637908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:42.637908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:42.637908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:42.637908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:42.637908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:42.637908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:42.637908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:42.637908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:42.637908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:42.637908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:42.637908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:42.637908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:42.637908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:42.637908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:42.637908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:42.638790 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:42.638790 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:42.638790 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:42.638790 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:42.650283 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 292/599] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:42.651278 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09650b16-e899-482e-8313-be6eb1273a41 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:42.699341 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349f7f3-c65d-4ff4-ba34-1bfae5534981 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:42.710893 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:42.753476 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0c102-6784-4a13-80d7-a5584584ec05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:42.755241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0c102-6784-4a13-80d7-a5584584ec05 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:42.756005 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0c102-6784-4a13-80d7-a5584584ec05 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:42.757104 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0c102-6784-4a13-80d7-a5584584ec05 None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:42.758159 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a0c102-6784-4a13-80d7-a5584584ec05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:42.758804 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a0c102-6784-4a13-80d7-a5584584ec05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:42.761629 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a0c102-6784-4a13-80d7-a5584584ec05 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:42.779601 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 308/600] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:42.790876 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0c102-6784-4a13-80d7-a5584584ec05 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:42.814421 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:42.824724 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09650b16-e899-482e-8313-be6eb1273a41 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:42.902971 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1905b-b12e-4780-abba-4cd1302a2a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:42.903513 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1905b-b12e-4780-abba-4cd1302a2a3d None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:42.903795 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1905b-b12e-4780-abba-4cd1302a2a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:42.904073 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1905b-b12e-4780-abba-4cd1302a2a3d None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:42.904778 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1905b-b12e-4780-abba-4cd1302a2a3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:42.905088 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1905b-b12e-4780-abba-4cd1302a2a3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:42.911002 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1905b-b12e-4780-abba-4cd1302a2a3d None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:42.926774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 293/601] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:42.932789 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1905b-b12e-4780-abba-4cd1302a2a3d None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:43.022809 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0c102-6784-4a13-80d7-a5584584ec05 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:43.074363 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6e2bf-05dd-4208-afa2-99a4d1c00c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:43.075065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6e2bf-05dd-4208-afa2-99a4d1c00c1e None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:43.077559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6e2bf-05dd-4208-afa2-99a4d1c00c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:43.077886 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6e2bf-05dd-4208-afa2-99a4d1c00c1e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:43.089258 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6e2bf-05dd-4208-afa2-99a4d1c00c1e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:43.090993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6e2bf-05dd-4208-afa2-99a4d1c00c1e None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:43.105004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-e4c6e2bf-05dd-4208-afa2-99a4d1c00c1e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:43.105004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:43.105004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:43.105004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:43.105004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:43.105004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:43.105004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:43.105004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:43.105004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:43.105004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:43.105004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:43.105004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:43.105004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:54:43.105004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:54:43.105004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:54:43.105928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:54:43.105928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:43.105928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:43.105928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:43.105928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:43.105928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:43.105928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:43.105928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:43.105928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:43.105928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:43.105928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:43.105928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:43.105928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:43.105928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:43.105928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:43.105928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:43.105928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:43.106832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:54:43.106832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:54:43.106832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:54:43.106832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:54:43.106832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:54:43.106832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:54:43.106832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:43.106832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:43.111119 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 309/602] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:54:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:43.113794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6e2bf-05dd-4208-afa2-99a4d1c00c1e None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:43.145940 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:43.190484 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1905b-b12e-4780-abba-4cd1302a2a3d None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:43.226072 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee83cf4e-289c-4255-940b-5b7836fb5058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:43.226723 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83cf4e-289c-4255-940b-5b7836fb5058 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:43.227053 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83cf4e-289c-4255-940b-5b7836fb5058 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:43.227329 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83cf4e-289c-4255-940b-5b7836fb5058 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:43.234979 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee83cf4e-289c-4255-940b-5b7836fb5058 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:43.235936 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6e2bf-05dd-4208-afa2-99a4d1c00c1e None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:43.237793 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee83cf4e-289c-4255-940b-5b7836fb5058 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:43.251696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-ee83cf4e-289c-4255-940b-5b7836fb5058 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:43.251696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:43.251696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:43.251696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:43.251696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:43.251696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:43.251696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:43.251696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:43.251696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:43.251696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:43.251696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:43.251696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:43.251696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:54:43.251696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:54:43.251696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:54:43.252620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:54:43.252620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:43.252620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:43.252620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:43.252620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:43.252620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:43.252620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:43.252620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:43.252620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:43.252620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:43.252620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:43.252620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:43.252620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:43.252620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:43.252620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:43.252620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:43.252620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:43.253505 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:54:43.253505 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:54:43.253505 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:54:43.253505 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:54:43.253505 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:54:43.253505 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:54:43.253505 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:43.253505 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:43.257502 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 294/603] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:54:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:43.258759 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00fc5f-8dcf-44b0-a717-e25c5aba5694 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:43.259070 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00fc5f-8dcf-44b0-a717-e25c5aba5694 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:43.259347 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00fc5f-8dcf-44b0-a717-e25c5aba5694 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:43.260848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 295/604] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:43.261546 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00fc5f-8dcf-44b0-a717-e25c5aba5694 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:43.275373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3ac00-0916-4a77-80eb-153159de0520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:43.276017 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3ac00-0916-4a77-80eb-153159de0520 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:43.276305 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3ac00-0916-4a77-80eb-153159de0520 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:43.276584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3ac00-0916-4a77-80eb-153159de0520 None admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:43.277719 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c3ac00-0916-4a77-80eb-153159de0520 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:43.278035 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c3ac00-0916-4a77-80eb-153159de0520 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:43.280025 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c3ac00-0916-4a77-80eb-153159de0520 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:43.285296 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:43.293628 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 310/605] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:54:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:43.295059 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a15598-ecf9-4368-bdc7-01d3ecfad740 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:43.295170 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a15598-ecf9-4368-bdc7-01d3ecfad740 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:43.295414 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a15598-ecf9-4368-bdc7-01d3ecfad740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:43.316074 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-39a15598-ecf9-4368-bdc7-01d3ecfad740 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:43.317655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-39a15598-ecf9-4368-bdc7-01d3ecfad740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:43.382990 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00fc5f-8dcf-44b0-a717-e25c5aba5694 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:43.394987 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 311/606] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:43.396372 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03904896-8c18-4c17-809b-44ac50eda0db None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:43.396702 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03904896-8c18-4c17-809b-44ac50eda0db None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:43.397088 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03904896-8c18-4c17-809b-44ac50eda0db None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:43.403032 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 312/607] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:43.403032 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03904896-8c18-4c17-809b-44ac50eda0db None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:43.424530 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:43.430975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4073a04-ee9f-40d9-b531-c2ec8e761d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:43.431681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4073a04-ee9f-40d9-b531-c2ec8e761d80 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:43.431988 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4073a04-ee9f-40d9-b531-c2ec8e761d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:43.437185 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4073a04-ee9f-40d9-b531-c2ec8e761d80 None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:43.437185 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4073a04-ee9f-40d9-b531-c2ec8e761d80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:43.437185 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4073a04-ee9f-40d9-b531-c2ec8e761d80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:43.437185 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4073a04-ee9f-40d9-b531-c2ec8e761d80 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:43.448662 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 296/608] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:54:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:43.449948 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be3aa0-93a0-43ad-a8e7-0dcae040d470 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:43.450341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be3aa0-93a0-43ad-a8e7-0dcae040d470 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:43.450730 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be3aa0-93a0-43ad-a8e7-0dcae040d470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:43.471223 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-e8be3aa0-93a0-43ad-a8e7-0dcae040d470 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:43.472523 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e8be3aa0-93a0-43ad-a8e7-0dcae040d470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:43.518070 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03904896-8c18-4c17-809b-44ac50eda0db None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:43.548090 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 297/609] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:43.549178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be3aa0-93a0-43ad-a8e7-0dcae040d470 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:43.559608 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c0fa6-9b6e-4dd5-bf98-9b8aa0a02ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:43.560533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c0fa6-9b6e-4dd5-bf98-9b8aa0a02ef9 None admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:43.561043 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c0fa6-9b6e-4dd5-bf98-9b8aa0a02ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:43.561537 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c0fa6-9b6e-4dd5-bf98-9b8aa0a02ef9 None admin] PATH_INFO: `/v3/projects/cf3763a37383456184a9ca9351b4ce03/users/cff15ea4cbc64b068aa5f6586b5bff87/roles/cae571342db14f5fac7849efe7f8636e` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:43.573099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:43.577478 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c0fa6-9b6e-4dd5-bf98-9b8aa0a02ef9 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf3763a37383456184a9ca9351b4ce03, user_id=cff15ea4cbc64b068aa5f6586b5bff87, role_id=cae571342db14f5fac7849efe7f8636e)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:43.579042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c0fa6-9b6e-4dd5-bf98-9b8aa0a02ef9 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:43.660305 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be3aa0-93a0-43ad-a8e7-0dcae040d470 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:43.689071 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 313/610] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 3 04:54:43 2022] PUT /identity/v3/projects/cf3763a37383456184a9ca9351b4ce03/users/cff15ea4cbc64b068aa5f6586b5bff87/roles/cae571342db14f5fac7849efe7f8636e => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 04:54:43.697975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12d04a-e505-437c-a7f2-44a3376b4c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:43.699128 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12d04a-e505-437c-a7f2-44a3376b4c40 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:43.699621 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12d04a-e505-437c-a7f2-44a3376b4c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:43.699805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c0fa6-9b6e-4dd5-bf98-9b8aa0a02ef9 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:43.700259 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12d04a-e505-437c-a7f2-44a3376b4c40 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:43.713741 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12d04a-e505-437c-a7f2-44a3376b4c40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:43.719848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-eb12d04a-e505-437c-a7f2-44a3376b4c40 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:43.719848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:43.719848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:43.719848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:43.719848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:43.719848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:43.719848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:43.719848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:43.719848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:43.719848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:43.719848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:43.719848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:43.719848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:43.719848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:43.719848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:43.719848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:43.721864 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:43.721864 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:43.721864 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:43.721864 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:43.721864 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:43.721864 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:43.721864 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:43.721864 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:43.721864 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:43.721864 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:43.721864 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:43.721864 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:43.721864 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:43.721864 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:43.721864 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:43.721864 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:43.722734 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:43.722734 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:43.722734 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:43.722734 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:43.722734 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:43.722734 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:43.722734 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:43.722734 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:43.722734 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:43.722734 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:43.722734 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:43.722734 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:43.727046 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 298/611] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 3 04:54:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:43.728359 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12d04a-e505-437c-a7f2-44a3376b4c40 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:43.824984 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:43.827809 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:43.870489 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12d04a-e505-437c-a7f2-44a3376b4c40 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:43.874247 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c0fa6-9b6e-4dd5-bf98-9b8aa0a02ef9 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:43.924165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478bb28-b34a-4e3c-9233-d6a93a6109f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:43.924857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478bb28-b34a-4e3c-9233-d6a93a6109f6 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:43.925144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478bb28-b34a-4e3c-9233-d6a93a6109f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:43.925457 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478bb28-b34a-4e3c-9233-d6a93a6109f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:43.926943 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9478d-b7b6-4239-a8de-6ed37b503aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:43.927625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9478d-b7b6-4239-a8de-6ed37b503aa8 None admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:43.927914 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9478d-b7b6-4239-a8de-6ed37b503aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:43.928180 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9478d-b7b6-4239-a8de-6ed37b503aa8 None admin] PATH_INFO: `/v3/projects/cf3763a37383456184a9ca9351b4ce03/users/84cc14b33895443b893fab487bfe1858/roles/cae571342db14f5fac7849efe7f8636e` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:43.939662 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c478bb28-b34a-4e3c-9233-d6a93a6109f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:43.944548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a9478d-b7b6-4239-a8de-6ed37b503aa8 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf3763a37383456184a9ca9351b4ce03, user_id=84cc14b33895443b893fab487bfe1858, role_id=cae571342db14f5fac7849efe7f8636e)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:43.945496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-c478bb28-b34a-4e3c-9233-d6a93a6109f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:43.945496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:43.945496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:43.945496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:43.945496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:43.945496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:43.945496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:43.945496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:43.945496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:43.945496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:43.945496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:43.945496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:43.945496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:43.945496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:43.945496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:43.945496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:43.946409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:43.946409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:43.946409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:43.946409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:43.946409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:43.946409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:43.946409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:43.946409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:43.946409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:43.946409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:43.946409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:43.946409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:43.946409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:43.946409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:43.946409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:43.946409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:43.947315 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:43.947315 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:43.947315 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:43.947315 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:43.947315 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:43.947315 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:43.947315 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:43.947315 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:43.947315 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:43.947315 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:43.947315 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:43.947315 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:43.947970 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a9478d-b7b6-4239-a8de-6ed37b503aa8 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:43.949804 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 299/612] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 3 04:54:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:43.955169 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478bb28-b34a-4e3c-9233-d6a93a6109f6 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:44.022875 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 314/613] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 3 04:54:43 2022] PUT /identity/v3/projects/cf3763a37383456184a9ca9351b4ce03/users/84cc14b33895443b893fab487bfe1858/roles/cae571342db14f5fac7849efe7f8636e => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 04:54:44.022875 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9478d-b7b6-4239-a8de-6ed37b503aa8 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:44.139608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478bb28-b34a-4e3c-9233-d6a93a6109f6 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:44.197114 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:44.210382 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9478d-b7b6-4239-a8de-6ed37b503aa8 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:44.218277 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fd10b-9e08-40c6-8989-eba92d78e700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:44.218934 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fd10b-9e08-40c6-8989-eba92d78e700 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:44.219213 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fd10b-9e08-40c6-8989-eba92d78e700 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:44.219482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fd10b-9e08-40c6-8989-eba92d78e700 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:44.220208 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43fd10b-9e08-40c6-8989-eba92d78e700 None admin] RBAC: Adding query filter params (name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:44.220510 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43fd10b-9e08-40c6-8989-eba92d78e700 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:44.221918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43fd10b-9e08-40c6-8989-eba92d78e700 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:44.236570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 300/614] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 3 04:54:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:44.237862 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:44.245058 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fd10b-9e08-40c6-8989-eba92d78e700 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:44.263616 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc35e0-8d3a-4174-a4d5-1e76d48cfa1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:44.264294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc35e0-8d3a-4174-a4d5-1e76d48cfa1d None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:44.265487 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc35e0-8d3a-4174-a4d5-1e76d48cfa1d None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:44.266131 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc35e0-8d3a-4174-a4d5-1e76d48cfa1d None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:44.267483 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc35e0-8d3a-4174-a4d5-1e76d48cfa1d None admin] RBAC: Adding query filter params (name=service) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:44.268033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc35e0-8d3a-4174-a4d5-1e76d48cfa1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:44.269578 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc35e0-8d3a-4174-a4d5-1e76d48cfa1d None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:44.280579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 315/615] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 3 04:54:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:44.288226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc35e0-8d3a-4174-a4d5-1e76d48cfa1d None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:44.380369 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fd10b-9e08-40c6-8989-eba92d78e700 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:44.408389 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc35e0-8d3a-4174-a4d5-1e76d48cfa1d None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:44.427509 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c7b22-eb11-4ca3-9979-75fa03a26907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:44.428792 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c7b22-eb11-4ca3-9979-75fa03a26907 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:44.429254 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c7b22-eb11-4ca3-9979-75fa03a26907 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:44.429728 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c7b22-eb11-4ca3-9979-75fa03a26907 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:44.445245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26c7b22-eb11-4ca3-9979-75fa03a26907 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:44.446583 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26c7b22-eb11-4ca3-9979-75fa03a26907 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:44.450944 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2012a6ed-561d-46f1-b9be-786777bb2339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:44.451976 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012a6ed-561d-46f1-b9be-786777bb2339 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:44.452490 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012a6ed-561d-46f1-b9be-786777bb2339 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:44.452943 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012a6ed-561d-46f1-b9be-786777bb2339 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:44.463176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-b26c7b22-eb11-4ca3-9979-75fa03a26907 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:44.463176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:44.463176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:44.463176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:44.463176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:44.463176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:44.463176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:44.463176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:44.463176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:44.463176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:44.463176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:44.463176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:44.463176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:44.463176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:44.463176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:44.464077 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:44.464077 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:44.464077 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:44.464077 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:44.464077 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:44.464077 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:44.464077 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:44.464077 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:44.464077 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:44.464077 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:44.464077 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:44.464077 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:44.464077 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:44.464077 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:44.464077 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:44.464077 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:44.464077 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:44.464962 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:44.464962 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:44.464962 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:44.464962 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:44.464962 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 301/616] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:44.468585 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2012a6ed-561d-46f1-b9be-786777bb2339 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:44.470250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2012a6ed-561d-46f1-b9be-786777bb2339 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:44.470362 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c7b22-eb11-4ca3-9979-75fa03a26907 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:44.488915 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-2012a6ed-561d-46f1-b9be-786777bb2339 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:44.488915 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:44.488915 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:44.488915 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:44.488915 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:44.488915 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:44.488915 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:44.488915 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:44.488915 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:44.488915 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:44.488915 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:44.488915 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:44.488915 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:44.488915 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:44.488915 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:44.489951 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:44.489951 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:44.489951 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:44.489951 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:44.489951 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:44.489951 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:44.489951 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:44.489951 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:44.489951 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:44.489951 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:44.489951 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:44.489951 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:44.489951 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:44.489951 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:44.489951 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:44.489951 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:44.489951 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:44.490929 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:44.490929 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:44.490929 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:44.490929 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:44.490929 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 316/617] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:44.498950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2012a6ed-561d-46f1-b9be-786777bb2339 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:44.549916 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:44.563208 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:44.628535 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c7b22-eb11-4ca3-9979-75fa03a26907 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:44.673658 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dea7da-b515-4bd1-a2cc-00b10b102e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:44.674281 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dea7da-b515-4bd1-a2cc-00b10b102e77 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:44.674564 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dea7da-b515-4bd1-a2cc-00b10b102e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:44.674862 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dea7da-b515-4bd1-a2cc-00b10b102e77 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:44.675560 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dea7da-b515-4bd1-a2cc-00b10b102e77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:44.675866 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dea7da-b515-4bd1-a2cc-00b10b102e77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:44.677241 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dea7da-b515-4bd1-a2cc-00b10b102e77 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:44.699296 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 302/618] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:44.704790 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dea7da-b515-4bd1-a2cc-00b10b102e77 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:44.716853 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2012a6ed-561d-46f1-b9be-786777bb2339 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:44.775948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f32ba-a55f-4325-bcd9-babc3cf0ad51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:44.776864 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f32ba-a55f-4325-bcd9-babc3cf0ad51 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:44.777155 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f32ba-a55f-4325-bcd9-babc3cf0ad51 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:44.777468 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f32ba-a55f-4325-bcd9-babc3cf0ad51 None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:44.778375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f32ba-a55f-4325-bcd9-babc3cf0ad51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:44.778741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f32ba-a55f-4325-bcd9-babc3cf0ad51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:44.781447 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f32ba-a55f-4325-bcd9-babc3cf0ad51 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:44.798787 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 317/619] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:44.800066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bf2c5-a285-45d7-a529-b1bc44f7c90d None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:44.800353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bf2c5-a285-45d7-a529-b1bc44f7c90d None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:44.800629 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bf2c5-a285-45d7-a529-b1bc44f7c90d None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:44.801884 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 318/620] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:44.806483 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8bf2c5-a285-45d7-a529-b1bc44f7c90d None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:44.837834 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:44.841497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dea7da-b515-4bd1-a2cc-00b10b102e77 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:44.881445 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d42e29c-8e37-4baa-9a7e-d8c290040ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:44.882091 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42e29c-8e37-4baa-9a7e-d8c290040ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:44.882377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42e29c-8e37-4baa-9a7e-d8c290040ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:44.882656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42e29c-8e37-4baa-9a7e-d8c290040ca7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:44.893143 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d42e29c-8e37-4baa-9a7e-d8c290040ca7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:44.895323 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d42e29c-8e37-4baa-9a7e-d8c290040ca7 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:44.901134 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:44.912097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-6d42e29c-8e37-4baa-9a7e-d8c290040ca7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 03 04:54:44.912097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:44.912097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:44.912097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:44.912097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:44.912097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:44.912097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:44.912097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:44.912097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:44.912097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:44.912097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:44.912097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:44.912097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:54:44.912097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:54:44.912097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:54:44.912097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:54:44.919810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:44.919810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:44.919810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:54:44.919810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:44.919810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:54:44.919810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:44.919810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:44.919810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:44.919810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:44.919810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:44.919810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:44.919810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:44.919810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:44.919810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:44.919810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:44.919810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:44.919810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:44.919810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:44.920745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:44.920745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:44.920745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:54:44.920745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:54:44.920745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:54:44.920745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:54:44.920745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:54:44.920745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:54:44.920745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 03 04:54:44.920745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:44.921874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8bf2c5-a285-45d7-a529-b1bc44f7c90d None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:44.921961 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 303/621] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:54:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:54:44.921961 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4604b-6eea-4e1b-a4b4-a2c6f6ee94f1 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:44.921961 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4604b-6eea-4e1b-a4b4-a2c6f6ee94f1 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:44.921961 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4604b-6eea-4e1b-a4b4-a2c6f6ee94f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:44.944461 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-40b4604b-6eea-4e1b-a4b4-a2c6f6ee94f1 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:44.945772 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-40b4604b-6eea-4e1b-a4b4-a2c6f6ee94f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:44.967033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ecedb-ba97-440a-8964-c404acc494d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:44.968815 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ecedb-ba97-440a-8964-c404acc494d3 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:44.969333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ecedb-ba97-440a-8964-c404acc494d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:44.969989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ecedb-ba97-440a-8964-c404acc494d3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:44.979752 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ecedb-ba97-440a-8964-c404acc494d3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:44.982075 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ecedb-ba97-440a-8964-c404acc494d3 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:44.998106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-6b1ecedb-ba97-440a-8964-c404acc494d3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 04:54:44.998106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:44.998106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:44.998106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:44.998106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:44.998106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:44.998106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:44.998106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:44.998106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:44.998106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:44.998106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:44.998106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:44.998106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:54:44.998106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:54:44.998106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:54:44.998106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:54:44.999461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:44.999461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:44.999461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:54:44.999461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:44.999461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:54:44.999461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:44.999461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:44.999461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:44.999461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:44.999461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:44.999461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:44.999461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:44.999461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:44.999461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:44.999461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:44.999461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:44.999461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:44.999461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:45.000439 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:45.000439 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:45.000439 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:54:45.000439 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:54:45.000439 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:54:45.000439 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:54:45.000439 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:54:45.000439 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:54:45.000439 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 04:54:45.000439 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:45.003872 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 319/622] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:54:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:45.005153 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ecedb-ba97-440a-8964-c404acc494d3 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:45.034833 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 304/623] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:45.036033 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b4604b-6eea-4e1b-a4b4-a2c6f6ee94f1 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:45.124777 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ecedb-ba97-440a-8964-c404acc494d3 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:45.130203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:45.140876 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b4604b-6eea-4e1b-a4b4-a2c6f6ee94f1 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:45.160999 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90386a3f-ca10-4150-a3aa-75100ebd950b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:45.161716 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90386a3f-ca10-4150-a3aa-75100ebd950b None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:45.162009 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90386a3f-ca10-4150-a3aa-75100ebd950b None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:45.162291 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90386a3f-ca10-4150-a3aa-75100ebd950b None admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:45.163115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90386a3f-ca10-4150-a3aa-75100ebd950b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:45.163428 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90386a3f-ca10-4150-a3aa-75100ebd950b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:45.165571 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90386a3f-ca10-4150-a3aa-75100ebd950b None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:45.179873 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f6e16-7405-47f4-9bbe-6ff24c7eba12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:45.180777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f6e16-7405-47f4-9bbe-6ff24c7eba12 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:45.181286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f6e16-7405-47f4-9bbe-6ff24c7eba12 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:45.181773 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f6e16-7405-47f4-9bbe-6ff24c7eba12 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:45.182760 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f6e16-7405-47f4-9bbe-6ff24c7eba12 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:45.183300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f6e16-7405-47f4-9bbe-6ff24c7eba12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:45.184933 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f6e16-7405-47f4-9bbe-6ff24c7eba12 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:45.185502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 320/624] 10.222.0.40 () {56 vars in 1135 bytes} [Wed Aug 3 04:54:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:45.190135 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90386a3f-ca10-4150-a3aa-75100ebd950b None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:45.203184 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:45.209088 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 305/625] 10.222.0.40 () {56 vars in 1137 bytes} [Wed Aug 3 04:54:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:45.210120 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f6e16-7405-47f4-9bbe-6ff24c7eba12 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:45.316404 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90386a3f-ca10-4150-a3aa-75100ebd950b None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:45.347462 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f6e16-7405-47f4-9bbe-6ff24c7eba12 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:45.354655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2fe43-9e02-48da-8afd-4f0b1b84a7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:45.355348 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2fe43-9e02-48da-8afd-4f0b1b84a7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:45.355665 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2fe43-9e02-48da-8afd-4f0b1b84a7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:45.356003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2fe43-9e02-48da-8afd-4f0b1b84a7e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:45.368083 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2fe43-9e02-48da-8afd-4f0b1b84a7e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:45.373135 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-ebf2fe43-9e02-48da-8afd-4f0b1b84a7e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:45.373135 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:45.373135 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:45.373135 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:45.373135 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:45.373135 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:45.373135 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:45.373135 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:45.373135 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:45.373135 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:45.373135 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:45.373135 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:45.373135 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:45.373135 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:45.373135 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:45.373135 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:45.374096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:45.374096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:45.374096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:45.374096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:45.374096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:45.374096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:45.374096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:45.374096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:45.374096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:45.374096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:45.374096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:45.374096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:45.374096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:45.374096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:45.374096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:45.374096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:45.374965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:45.374965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:45.374965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:45.374965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:45.374965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:45.374965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:45.374965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:45.374965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:45.374965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:45.374965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:45.374965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:45.374965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:45.374965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 321/626] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 3 04:54:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:45.376148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2fe43-9e02-48da-8afd-4f0b1b84a7e0 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:45.389579 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaeb6d3-aad3-4698-b8ae-12e6c7251421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:45.391293 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaeb6d3-aad3-4698-b8ae-12e6c7251421 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:45.391894 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaeb6d3-aad3-4698-b8ae-12e6c7251421 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:45.392388 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaeb6d3-aad3-4698-b8ae-12e6c7251421 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:45.404253 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaeb6d3-aad3-4698-b8ae-12e6c7251421 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:45.406731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaeb6d3-aad3-4698-b8ae-12e6c7251421 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:45.417182 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:45.424847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-ffaeb6d3-aad3-4698-b8ae-12e6c7251421 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:45.424847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:45.424847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:45.424847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:45.424847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:45.424847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:45.424847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:45.424847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:45.424847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:45.424847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:45.424847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:45.424847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:45.424847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:45.424847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:45.424847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:45.425823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:45.425823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:45.425823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:45.425823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:45.425823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:45.425823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:45.425823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:45.425823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:45.425823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:45.425823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:45.425823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:45.425823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:45.425823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:45.425823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:45.425823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:45.425823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:45.425823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:45.426764 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:45.426764 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:45.426764 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:45.426764 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:45.433685 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 306/627] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:45.435286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaeb6d3-aad3-4698-b8ae-12e6c7251421 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:45.455435 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:45.500818 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2fe43-9e02-48da-8afd-4f0b1b84a7e0 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:45.539644 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fee927-715d-447b-bf0c-9001ca144bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:45.540279 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fee927-715d-447b-bf0c-9001ca144bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:45.540562 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fee927-715d-447b-bf0c-9001ca144bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:45.540839 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fee927-715d-447b-bf0c-9001ca144bb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:45.575513 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaeb6d3-aad3-4698-b8ae-12e6c7251421 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:45.612554 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e0e96-cc53-41ff-aafa-c1c5fc126c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:45.613620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e0e96-cc53-41ff-aafa-c1c5fc126c62 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:45.614191 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e0e96-cc53-41ff-aafa-c1c5fc126c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:45.614672 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e0e96-cc53-41ff-aafa-c1c5fc126c62 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:45.615614 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e0e96-cc53-41ff-aafa-c1c5fc126c62 None admin] RBAC: Adding query filter params (name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:45.616125 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e0e96-cc53-41ff-aafa-c1c5fc126c62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:45.616583 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fee927-715d-447b-bf0c-9001ca144bb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:45.617938 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e0e96-cc53-41ff-aafa-c1c5fc126c62 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:45.618123 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fee927-715d-447b-bf0c-9001ca144bb3 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:45.628092 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 307/628] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 3 04:54:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:45.629236 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e0e96-cc53-41ff-aafa-c1c5fc126c62 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:45.691935 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:45.705004 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:45.736125 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 322/629] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:54:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:54:45.737048 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fee927-715d-447b-bf0c-9001ca144bb3 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:45.747608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e0e96-cc53-41ff-aafa-c1c5fc126c62 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:45.785774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7caf05e-6dfb-4e17-a4d0-322fdf9363b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:45.786771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7caf05e-6dfb-4e17-a4d0-322fdf9363b7 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:45.787382 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7caf05e-6dfb-4e17-a4d0-322fdf9363b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:45.787877 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7caf05e-6dfb-4e17-a4d0-322fdf9363b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:45.788838 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7caf05e-6dfb-4e17-a4d0-322fdf9363b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:45.789405 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7caf05e-6dfb-4e17-a4d0-322fdf9363b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:45.791024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7caf05e-6dfb-4e17-a4d0-322fdf9363b7 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:45.809713 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 308/630] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:45.811097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7caf05e-6dfb-4e17-a4d0-322fdf9363b7 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:45.848583 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fee927-715d-447b-bf0c-9001ca144bb3 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:45.888370 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d674a-75b9-41ce-935c-3eee9f04d0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:45.889027 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d674a-75b9-41ce-935c-3eee9f04d0f9 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:45.889312 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d674a-75b9-41ce-935c-3eee9f04d0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:45.889626 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d674a-75b9-41ce-935c-3eee9f04d0f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:45.901439 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d674a-75b9-41ce-935c-3eee9f04d0f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:45.903712 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d674a-75b9-41ce-935c-3eee9f04d0f9 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:45.923146 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-375d674a-75b9-41ce-935c-3eee9f04d0f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:45.923146 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:45.923146 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:45.923146 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:45.923146 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:45.923146 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:45.923146 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:45.923146 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:45.923146 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:45.923146 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:45.923146 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:45.923146 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:45.923146 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:45.923146 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:45.923146 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:45.925113 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:45.925113 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:45.925113 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:45.925113 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:45.925113 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:45.925113 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:45.925113 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:45.925113 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:45.925113 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:45.925113 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:45.925113 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:45.925113 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:45.925113 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:45.925113 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:45.925113 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:45.925113 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:45.925113 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:45.926041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:45.926041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:45.926041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:45.926041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:45.926041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 323/631] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:45.926041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d674a-75b9-41ce-935c-3eee9f04d0f9 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:45.948616 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7caf05e-6dfb-4e17-a4d0-322fdf9363b7 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:45.954413 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:45.974119 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:45.985434 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25f112-8642-4509-9d86-ccff0c0daf42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:45.986080 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25f112-8642-4509-9d86-ccff0c0daf42 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:45.986369 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25f112-8642-4509-9d86-ccff0c0daf42 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:45.986647 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25f112-8642-4509-9d86-ccff0c0daf42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:45.997185 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef25f112-8642-4509-9d86-ccff0c0daf42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:45.998770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef25f112-8642-4509-9d86-ccff0c0daf42 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:46.015758 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-ef25f112-8642-4509-9d86-ccff0c0daf42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:46.015758 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:46.015758 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:46.015758 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:46.015758 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:46.015758 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:46.015758 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:46.015758 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:46.015758 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:46.015758 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:46.015758 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:46.015758 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:46.015758 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:46.015758 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:46.015758 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:46.016719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:46.016719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:46.016719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:46.016719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:46.016719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:46.016719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:46.016719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:46.016719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:46.016719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:46.016719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:46.016719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:46.016719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:46.016719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:46.016719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:46.016719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:46.016719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:46.016719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:46.017605 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:46.017605 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:46.017605 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:46.017605 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:46.017605 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 309/632] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:46.018310 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25f112-8642-4509-9d86-ccff0c0daf42 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:46.042218 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d674a-75b9-41ce-935c-3eee9f04d0f9 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:46.077081 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0c4ce-c859-4f7d-a1ae-a6b2bd55527b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:46.078254 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0c4ce-c859-4f7d-a1ae-a6b2bd55527b None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:46.078541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0c4ce-c859-4f7d-a1ae-a6b2bd55527b None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:46.078828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0c4ce-c859-4f7d-a1ae-a6b2bd55527b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:46.085565 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0c4ce-c859-4f7d-a1ae-a6b2bd55527b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:46.087055 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0c4ce-c859-4f7d-a1ae-a6b2bd55527b None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:46.100041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-06d0c4ce-c859-4f7d-a1ae-a6b2bd55527b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:46.100041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:46.100041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:46.100041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:46.100041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:46.100041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:46.100041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:46.100041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:46.100041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:46.100041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:46.100041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:46.100041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:46.100041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:54:46.100041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:54:46.100041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:54:46.101154 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:54:46.101154 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:46.101154 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:46.101154 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:46.101154 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:46.101154 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:46.101154 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:46.101154 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:46.101154 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:46.101154 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:46.101154 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:46.101154 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:46.101154 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:46.101154 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:46.101154 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:46.101154 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:46.101154 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:46.102038 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:54:46.102038 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:54:46.102038 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:54:46.102038 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:54:46.102038 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:54:46.102038 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:54:46.102038 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:46.102038 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:46.102038 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 324/633] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:54:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:46.102532 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fd002-f96d-47f7-919a-27254ed1eb47 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:46.102664 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fd002-f96d-47f7-919a-27254ed1eb47 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:46.102972 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fd002-f96d-47f7-919a-27254ed1eb47 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:46.104364 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 325/634] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:46.104968 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fd002-f96d-47f7-919a-27254ed1eb47 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:46.130399 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25f112-8642-4509-9d86-ccff0c0daf42 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:46.167359 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9119b-4051-47e7-96c1-bc73202cb13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:46.167975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9119b-4051-47e7-96c1-bc73202cb13d None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:46.168260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9119b-4051-47e7-96c1-bc73202cb13d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:46.168537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9119b-4051-47e7-96c1-bc73202cb13d None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:46.169274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9119b-4051-47e7-96c1-bc73202cb13d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:46.169525 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9119b-4051-47e7-96c1-bc73202cb13d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:46.170947 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9119b-4051-47e7-96c1-bc73202cb13d None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:46.187069 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 310/635] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:46.187069 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dba0d8-058d-4200-b60f-d842e9bad3fc None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:46.187069 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dba0d8-058d-4200-b60f-d842e9bad3fc None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:46.187069 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dba0d8-058d-4200-b60f-d842e9bad3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:46.195376 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:46.203170 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-39dba0d8-058d-4200-b60f-d842e9bad3fc None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:46.204381 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-39dba0d8-058d-4200-b60f-d842e9bad3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:46.217334 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fd002-f96d-47f7-919a-27254ed1eb47 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:46.232164 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:46.256513 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56bd5cb-0fbd-4c16-939a-9d95bc8c0895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:46.257151 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bd5cb-0fbd-4c16-939a-9d95bc8c0895 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:46.257436 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bd5cb-0fbd-4c16-939a-9d95bc8c0895 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:46.257716 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bd5cb-0fbd-4c16-939a-9d95bc8c0895 None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:46.258420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56bd5cb-0fbd-4c16-939a-9d95bc8c0895 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:46.258717 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56bd5cb-0fbd-4c16-939a-9d95bc8c0895 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:46.260142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56bd5cb-0fbd-4c16-939a-9d95bc8c0895 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:46.273299 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 326/636] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:46.274514 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56bd5cb-0fbd-4c16-939a-9d95bc8c0895 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:46.278858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 311/637] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:46.280022 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dba0d8-058d-4200-b60f-d842e9bad3fc None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:46.385500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56bd5cb-0fbd-4c16-939a-9d95bc8c0895 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:46.388870 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dba0d8-058d-4200-b60f-d842e9bad3fc None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:46.419465 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63b5cf-36a0-41d6-8370-25dbd02df651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:46.420079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63b5cf-36a0-41d6-8370-25dbd02df651 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:46.420362 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63b5cf-36a0-41d6-8370-25dbd02df651 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:46.420636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63b5cf-36a0-41d6-8370-25dbd02df651 None admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:46.421552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee63b5cf-36a0-41d6-8370-25dbd02df651 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:46.421856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee63b5cf-36a0-41d6-8370-25dbd02df651 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:46.423259 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62234d29-c0a9-42c0-bcbb-fa8698685b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:46.423466 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee63b5cf-36a0-41d6-8370-25dbd02df651 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:46.423900 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62234d29-c0a9-42c0-bcbb-fa8698685b4e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:46.424185 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62234d29-c0a9-42c0-bcbb-fa8698685b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:46.424464 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62234d29-c0a9-42c0-bcbb-fa8698685b4e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:46.431791 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62234d29-c0a9-42c0-bcbb-fa8698685b4e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:46.431964 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:46.433236 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62234d29-c0a9-42c0-bcbb-fa8698685b4e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:46.435285 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 327/638] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:54:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:46.436072 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63b5cf-36a0-41d6-8370-25dbd02df651 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:46.445990 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-62234d29-c0a9-42c0-bcbb-fa8698685b4e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 04:54:46.445990 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:46.445990 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:46.445990 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:46.445990 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:46.445990 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:46.445990 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:46.445990 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:46.445990 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:46.445990 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:46.445990 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:46.445990 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:46.445990 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:54:46.445990 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:54:46.445990 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:54:46.445990 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:54:46.448459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:46.448459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:46.448459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:54:46.448459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:46.448459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:54:46.448459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:46.448459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:46.448459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:46.448459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:46.448459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:46.448459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:46.448459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:46.448459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:46.448459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:46.448459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:46.448459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:46.448459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:46.448459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:46.449649 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:46.449649 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:46.449649 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:54:46.449649 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:54:46.449649 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:54:46.449649 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:54:46.449649 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:54:46.449649 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:54:46.449649 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 04:54:46.449649 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:46.449649 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 312/639] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Aug 3 04:54:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:46.449649 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62234d29-c0a9-42c0-bcbb-fa8698685b4e None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:46.474529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:46.540567 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63b5cf-36a0-41d6-8370-25dbd02df651 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:46.552512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62234d29-c0a9-42c0-bcbb-fa8698685b4e None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:46.573666 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e1e318-2b43-466f-9179-9a99a05dac00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:46.574299 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1e318-2b43-466f-9179-9a99a05dac00 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:46.574584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1e318-2b43-466f-9179-9a99a05dac00 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:46.574877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1e318-2b43-466f-9179-9a99a05dac00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:46.581737 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e1e318-2b43-466f-9179-9a99a05dac00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:46.583228 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e1e318-2b43-466f-9179-9a99a05dac00 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:46.585488 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd00665-a0e0-4f9a-bc4d-cb5e1aff1fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:46.586124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd00665-a0e0-4f9a-bc4d-cb5e1aff1fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:46.586409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd00665-a0e0-4f9a-bc4d-cb5e1aff1fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:46.586690 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd00665-a0e0-4f9a-bc4d-cb5e1aff1fc8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:46.596066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-b5e1e318-2b43-466f-9179-9a99a05dac00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:46.596066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:46.596066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:46.596066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:46.596066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:46.596066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:46.596066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:46.596066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:46.596066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:46.596066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:46.596066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:46.596066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:46.596066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:54:46.596066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:54:46.596066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:54:46.597115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:54:46.597115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:46.597115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:46.597115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:46.597115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:46.597115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:46.597115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:46.597115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:46.597115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:46.597115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:46.597115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:46.597115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:46.597115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:46.597115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:46.597115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:46.597115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:46.597115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:46.598015 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:54:46.598015 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:54:46.598015 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:54:46.598015 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:54:46.598015 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:54:46.598015 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:54:46.598015 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:46.598015 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:46.598015 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 328/640] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:54:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:46.598536 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd00665-a0e0-4f9a-bc4d-cb5e1aff1fc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:46.598610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f6acc-ed77-439a-92ca-cc6972125e36 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:46.598610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f6acc-ed77-439a-92ca-cc6972125e36 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:46.599043 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f6acc-ed77-439a-92ca-cc6972125e36 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:46.600282 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 329/641] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:46.600795 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f6acc-ed77-439a-92ca-cc6972125e36 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:46.606407 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-3cd00665-a0e0-4f9a-bc4d-cb5e1aff1fc8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:46.606407 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:46.606407 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:46.606407 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:46.606407 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:46.606407 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:46.606407 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:46.606407 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:46.606407 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:46.606407 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:46.606407 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:46.606407 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:46.606407 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:46.606407 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:46.606407 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:46.606407 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:46.607569 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:46.607569 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:46.607569 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:46.607569 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:46.607569 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:46.607569 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:46.607569 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:46.607569 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:46.607569 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:46.607569 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:46.607569 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:46.607569 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:46.607569 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:46.607569 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:46.607569 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:46.607569 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:46.613891 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:46.613891 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:46.613891 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:46.613891 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:46.613891 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:46.613891 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:46.613891 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:46.613891 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:46.613891 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:46.613891 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:46.613891 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:46.613891 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:46.613891 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 313/642] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 3 04:54:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:46.613891 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd00665-a0e0-4f9a-bc4d-cb5e1aff1fc8 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:46.669550 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:46.699232 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:46.712559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f6acc-ed77-439a-92ca-cc6972125e36 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:46.715096 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd00665-a0e0-4f9a-bc4d-cb5e1aff1fc8 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:46.746504 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65243759-89ea-402b-8543-b1f14826021e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:46.747275 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65243759-89ea-402b-8543-b1f14826021e None admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:46.747564 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65243759-89ea-402b-8543-b1f14826021e None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:46.747843 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65243759-89ea-402b-8543-b1f14826021e None admin] PATH_INFO: `/v3/projects/cf3763a37383456184a9ca9351b4ce03/users/e1014bcb4d36441aa46042941c9a63d0/roles/cae571342db14f5fac7849efe7f8636e` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:46.749340 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8914584e-d0dc-43b4-8398-0d8af24c733b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:46.749944 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914584e-d0dc-43b4-8398-0d8af24c733b None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:46.750230 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914584e-d0dc-43b4-8398-0d8af24c733b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:46.750506 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914584e-d0dc-43b4-8398-0d8af24c733b None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:46.751350 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8914584e-d0dc-43b4-8398-0d8af24c733b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:46.751654 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8914584e-d0dc-43b4-8398-0d8af24c733b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:46.757570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8914584e-d0dc-43b4-8398-0d8af24c733b None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:46.762356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65243759-89ea-402b-8543-b1f14826021e None admin] RBAC: Authorizing `identity:create_grant(project_id=cf3763a37383456184a9ca9351b4ce03, user_id=e1014bcb4d36441aa46042941c9a63d0, role_id=cae571342db14f5fac7849efe7f8636e)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:46.763964 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65243759-89ea-402b-8543-b1f14826021e None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:46.782674 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 314/643] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Aug 3 04:54:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:46.783600 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8914584e-d0dc-43b4-8398-0d8af24c733b None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:46.813642 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 330/644] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 3 04:54:46 2022] PUT /identity/v3/projects/cf3763a37383456184a9ca9351b4ce03/users/e1014bcb4d36441aa46042941c9a63d0/roles/cae571342db14f5fac7849efe7f8636e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 04:54:46.814972 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3a49e-0d2c-4f0f-af6f-6069748e81b1 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:46.815268 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3a49e-0d2c-4f0f-af6f-6069748e81b1 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:46.815552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3a49e-0d2c-4f0f-af6f-6069748e81b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:46.836077 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-b7b3a49e-0d2c-4f0f-af6f-6069748e81b1 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:46.837371 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-b7b3a49e-0d2c-4f0f-af6f-6069748e81b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:46.897794 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8914584e-d0dc-43b4-8398-0d8af24c733b None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:46.918094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 331/645] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:46.918832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b3a49e-0d2c-4f0f-af6f-6069748e81b1 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:46.963219 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b096c8-e603-47ba-9531-407a5c3114ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:46.963925 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b096c8-e603-47ba-9531-407a5c3114ba None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:46.964131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b096c8-e603-47ba-9531-407a5c3114ba None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:46.964409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b096c8-e603-47ba-9531-407a5c3114ba None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:46.965782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b096c8-e603-47ba-9531-407a5c3114ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:46.966090 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b096c8-e603-47ba-9531-407a5c3114ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:46.969069 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b096c8-e603-47ba-9531-407a5c3114ba None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:46.978382 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:46.984081 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 315/646] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:54:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:46.993620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b096c8-e603-47ba-9531-407a5c3114ba None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:46.993620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:47.044887 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b3a49e-0d2c-4f0f-af6f-6069748e81b1 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:47.083386 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407d5fa-0b61-43ed-a4be-0d5529ea9080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:47.084336 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407d5fa-0b61-43ed-a4be-0d5529ea9080 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:47.084336 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407d5fa-0b61-43ed-a4be-0d5529ea9080 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:47.084569 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407d5fa-0b61-43ed-a4be-0d5529ea9080 None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:47.085263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407d5fa-0b61-43ed-a4be-0d5529ea9080 None admin] RBAC: Adding query filter params (name=service) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:47.085572 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407d5fa-0b61-43ed-a4be-0d5529ea9080 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:47.086973 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407d5fa-0b61-43ed-a4be-0d5529ea9080 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:47.095486 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 332/647] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 3 04:54:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:47.098054 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b096c8-e603-47ba-9531-407a5c3114ba None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:47.100657 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407d5fa-0b61-43ed-a4be-0d5529ea9080 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:47.134388 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5080b3-ebe9-4426-82c4-79a980a993ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:47.135039 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5080b3-ebe9-4426-82c4-79a980a993ab None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:47.135320 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5080b3-ebe9-4426-82c4-79a980a993ab None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:47.135597 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5080b3-ebe9-4426-82c4-79a980a993ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:47.210127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5080b3-ebe9-4426-82c4-79a980a993ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:47.211552 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5080b3-ebe9-4426-82c4-79a980a993ab None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:47.213404 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:47.231135 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407d5fa-0b61-43ed-a4be-0d5529ea9080 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:47.233393 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:47.269148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20749067-4d8e-495f-8e2d-7cea5ed3f217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:47.269895 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20749067-4d8e-495f-8e2d-7cea5ed3f217 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:47.270367 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20749067-4d8e-495f-8e2d-7cea5ed3f217 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:47.270869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20749067-4d8e-495f-8e2d-7cea5ed3f217 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:47.282674 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20749067-4d8e-495f-8e2d-7cea5ed3f217 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:47.288065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-20749067-4d8e-495f-8e2d-7cea5ed3f217 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:47.288065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:47.288065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:47.288065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:47.288065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:47.288065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:47.288065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:47.288065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:47.288065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:47.288065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:47.288065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:47.288065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:47.288065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:47.288065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:47.288065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:47.288065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:47.289042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:47.289042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:47.289042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:47.289042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:47.289042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:47.289042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:47.289042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:47.289042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:47.289042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:47.289042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:47.289042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:47.289042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:47.289042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:47.289042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:47.289042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:47.289042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:47.289895 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:47.289895 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:47.289895 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:47.289895 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:47.289895 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:47.289895 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:47.289895 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:47.289895 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:47.289895 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:47.289895 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:47.289895 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:47.289895 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:47.289895 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 333/648] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 3 04:54:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:47.290591 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20749067-4d8e-495f-8e2d-7cea5ed3f217 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:47.321587 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 316/649] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:54:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:54:47.322252 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5080b3-ebe9-4426-82c4-79a980a993ab None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:47.402310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20749067-4d8e-495f-8e2d-7cea5ed3f217 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:47.438886 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599f983-5abd-4103-b8a2-47cf45f4e425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:47.439512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5080b3-ebe9-4426-82c4-79a980a993ab None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:47.439649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599f983-5abd-4103-b8a2-47cf45f4e425 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:47.439831 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599f983-5abd-4103-b8a2-47cf45f4e425 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:47.440058 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599f983-5abd-4103-b8a2-47cf45f4e425 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:47.440829 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599f983-5abd-4103-b8a2-47cf45f4e425 None admin] RBAC: Adding query filter params (role.id=cae571342db14f5fac7849efe7f8636e, scope.project.id=cf3763a37383456184a9ca9351b4ce03, user.id=c81ca55d52244bf1bfe4e829b6880c20) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:47.441231 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:47.441346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599f983-5abd-4103-b8a2-47cf45f4e425 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:47.443605 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599f983-5abd-4103-b8a2-47cf45f4e425 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:47.457444 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 334/650] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:54:47 2022] GET /identity/v3/role_assignments?user.id=c81ca55d52244bf1bfe4e829b6880c20&scope.project.id=cf3763a37383456184a9ca9351b4ce03&role.id=cae571342db14f5fac7849efe7f8636e => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:47.457444 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599f983-5abd-4103-b8a2-47cf45f4e425 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:47.478084 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba037b9-d9fc-4a9d-a035-8723e83f879c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:47.480624 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba037b9-d9fc-4a9d-a035-8723e83f879c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:47.480918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba037b9-d9fc-4a9d-a035-8723e83f879c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:47.481205 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba037b9-d9fc-4a9d-a035-8723e83f879c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:47.496158 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba037b9-d9fc-4a9d-a035-8723e83f879c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:47.496843 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:47.499695 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba037b9-d9fc-4a9d-a035-8723e83f879c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:47.516147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-1ba037b9-d9fc-4a9d-a035-8723e83f879c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:47.516147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:47.516147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:47.516147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:47.516147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:47.516147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:47.516147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:47.516147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:47.516147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:47.516147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:47.516147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:47.516147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:47.516147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:47.516147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:47.516147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:47.517314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:47.517314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:47.517314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:47.517314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:47.517314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:47.517314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:47.517314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:47.517314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:47.517314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:47.517314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:47.517314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:47.517314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:47.517314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:47.517314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:47.517314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:47.517314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:47.517314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:47.518633 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:47.518633 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:47.518633 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:47.518633 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:47.518633 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 317/651] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:47.518633 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba037b9-d9fc-4a9d-a035-8723e83f879c None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:47.606165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599f983-5abd-4103-b8a2-47cf45f4e425 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:47.634540 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba037b9-d9fc-4a9d-a035-8723e83f879c None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:47.650104 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac39be3-e22f-4446-9a6d-da99117a60b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:47.650728 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac39be3-e22f-4446-9a6d-da99117a60b5 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:47.651041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac39be3-e22f-4446-9a6d-da99117a60b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:47.651311 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac39be3-e22f-4446-9a6d-da99117a60b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:47.652039 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac39be3-e22f-4446-9a6d-da99117a60b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:47.652328 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac39be3-e22f-4446-9a6d-da99117a60b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:47.653702 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac39be3-e22f-4446-9a6d-da99117a60b5 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:47.664113 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 335/652] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 3 04:54:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:47.666497 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac39be3-e22f-4446-9a6d-da99117a60b5 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:47.682897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d899899-50df-496f-b646-8c04592831c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:47.684167 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d899899-50df-496f-b646-8c04592831c2 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:47.684453 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d899899-50df-496f-b646-8c04592831c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:47.684732 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d899899-50df-496f-b646-8c04592831c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:47.697416 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d899899-50df-496f-b646-8c04592831c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:47.699317 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d899899-50df-496f-b646-8c04592831c2 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:47.715930 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-8d899899-50df-496f-b646-8c04592831c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:47.715930 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:47.715930 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:47.715930 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:47.715930 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:47.715930 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:47.715930 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:47.715930 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:47.715930 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:47.715930 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:47.715930 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:47.715930 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:47.715930 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:47.715930 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:47.715930 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:47.716908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:47.716908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:47.716908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:47.716908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:47.716908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:47.716908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:47.716908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:47.716908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:47.716908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:47.716908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:47.716908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:47.716908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:47.716908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:47.716908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:47.716908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:47.716908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:47.716908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:47.718366 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:47.718366 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:47.718366 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:47.718366 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:47.718366 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 318/653] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:47.718366 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d899899-50df-496f-b646-8c04592831c2 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:47.727998 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:47.784930 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:47.798738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac39be3-e22f-4446-9a6d-da99117a60b5 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:47.836960 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d759a7-2f38-43b4-bf89-6619ecf8e8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:47.837604 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d759a7-2f38-43b4-bf89-6619ecf8e8dd None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:47.837892 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d759a7-2f38-43b4-bf89-6619ecf8e8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:47.838174 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d759a7-2f38-43b4-bf89-6619ecf8e8dd None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:47.839027 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d759a7-2f38-43b4-bf89-6619ecf8e8dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:47.839344 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d759a7-2f38-43b4-bf89-6619ecf8e8dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:47.840882 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d759a7-2f38-43b4-bf89-6619ecf8e8dd None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:47.859598 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 336/654] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:47.860766 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d759a7-2f38-43b4-bf89-6619ecf8e8dd None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:47.868248 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d899899-50df-496f-b646-8c04592831c2 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:47.918795 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b44b32-b621-4f3e-ae95-273cdda23551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:47.919544 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b44b32-b621-4f3e-ae95-273cdda23551 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:47.919829 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b44b32-b621-4f3e-ae95-273cdda23551 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:47.920102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b44b32-b621-4f3e-ae95-273cdda23551 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:47.933066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b44b32-b621-4f3e-ae95-273cdda23551 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:47.934752 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b44b32-b621-4f3e-ae95-273cdda23551 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:47.951367 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-f9b44b32-b621-4f3e-ae95-273cdda23551 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:47.951367 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:47.951367 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:47.951367 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:47.951367 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:47.951367 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:47.951367 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:47.951367 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:47.951367 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:47.951367 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:47.951367 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:47.951367 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:47.951367 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:47.951367 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:47.951367 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:47.952561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:47.952561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:47.952561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:47.952561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:47.952561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:47.952561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:47.952561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:47.952561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:47.952561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:47.952561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:47.952561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:47.952561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:47.952561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:47.952561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:47.952561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:47.952561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:47.952561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:47.953607 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:47.953607 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:47.953607 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:47.953607 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:47.955106 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 319/655] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:47.956003 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b44b32-b621-4f3e-ae95-273cdda23551 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:48.004217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d759a7-2f38-43b4-bf89-6619ecf8e8dd None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:48.038555 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:48.051307 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5d332-0cda-4e20-bbb4-36d28d855ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:48.052480 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5d332-0cda-4e20-bbb4-36d28d855ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:48.056282 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5d332-0cda-4e20-bbb4-36d28d855ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:48.056282 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5d332-0cda-4e20-bbb4-36d28d855ce2 None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:48.056282 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba5d332-0cda-4e20-bbb4-36d28d855ce2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:48.056282 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba5d332-0cda-4e20-bbb4-36d28d855ce2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:48.056622 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba5d332-0cda-4e20-bbb4-36d28d855ce2 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:48.065092 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b44b32-b621-4f3e-ae95-273cdda23551 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:48.074993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 337/656] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:48.076164 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5d332-0cda-4e20-bbb4-36d28d855ce2 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:48.112361 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f58604-54d4-490c-8c10-c3af4fd7e563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:48.113013 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f58604-54d4-490c-8c10-c3af4fd7e563 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:48.113302 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f58604-54d4-490c-8c10-c3af4fd7e563 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:48.113582 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f58604-54d4-490c-8c10-c3af4fd7e563 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:48.115195 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:48.123815 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f58604-54d4-490c-8c10-c3af4fd7e563 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:48.125693 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f58604-54d4-490c-8c10-c3af4fd7e563 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:48.138024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-f0f58604-54d4-490c-8c10-c3af4fd7e563 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 03 04:54:48.138024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:48.138024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:48.138024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:48.138024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:48.138024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:48.138024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:48.138024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:48.138024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:48.138024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:48.138024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:48.138024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:48.138024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:54:48.138024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:54:48.138024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:54:48.138024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:54:48.138995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:48.138995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:48.138995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:54:48.138995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:48.138995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:54:48.138995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:48.138995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:48.138995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:48.138995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:48.138995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:48.138995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:48.138995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:48.138995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:48.138995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:48.138995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:48.138995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:48.138995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:48.138995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:48.139999 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:48.139999 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:48.139999 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:54:48.139999 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:54:48.139999 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:54:48.139999 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:54:48.139999 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:54:48.139999 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:54:48.139999 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 03 04:54:48.139999 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:48.142597 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 320/657] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 3 04:54:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:48.143605 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f58604-54d4-490c-8c10-c3af4fd7e563 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:48.191388 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5d332-0cda-4e20-bbb4-36d28d855ce2 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:48.241601 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca15ec3-1c13-4ca3-ba8c-ac21aaecb031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:48.243088 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca15ec3-1c13-4ca3-ba8c-ac21aaecb031 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:48.243784 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca15ec3-1c13-4ca3-ba8c-ac21aaecb031 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:48.244483 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca15ec3-1c13-4ca3-ba8c-ac21aaecb031 None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:48.245870 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca15ec3-1c13-4ca3-ba8c-ac21aaecb031 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:48.246620 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca15ec3-1c13-4ca3-ba8c-ac21aaecb031 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:48.249471 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca15ec3-1c13-4ca3-ba8c-ac21aaecb031 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:48.268053 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 338/658] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:48.269604 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f58604-54d4-490c-8c10-c3af4fd7e563 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:48.269948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca15ec3-1c13-4ca3-ba8c-ac21aaecb031 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:48.289196 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:48.306475 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef632dd-ef04-449a-ba06-084c715bad15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:48.307156 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef632dd-ef04-449a-ba06-084c715bad15 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:48.307443 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef632dd-ef04-449a-ba06-084c715bad15 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:48.313265 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef632dd-ef04-449a-ba06-084c715bad15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:48.314711 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef632dd-ef04-449a-ba06-084c715bad15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:48.316404 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef632dd-ef04-449a-ba06-084c715bad15 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:48.328730 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-2ef632dd-ef04-449a-ba06-084c715bad15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:48.328730 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:48.328730 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:48.328730 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:48.328730 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:48.328730 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:48.328730 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:48.328730 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:48.328730 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:48.328730 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:48.328730 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:48.328730 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:48.328730 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:54:48.328730 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:54:48.328730 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:54:48.329681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:54:48.329681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:48.329681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:48.329681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:48.329681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:48.329681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:48.329681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:48.329681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:48.329681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:48.329681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:48.329681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:48.329681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:48.329681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:48.329681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:48.329681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:48.329681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:48.329681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:48.330590 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:54:48.330590 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:54:48.330590 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:54:48.330590 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:54:48.330590 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:54:48.330590 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:54:48.330590 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:48.330590 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:48.334982 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 321/659] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:54:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:48.336243 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef632dd-ef04-449a-ba06-084c715bad15 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:48.355166 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:48.391482 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca15ec3-1c13-4ca3-ba8c-ac21aaecb031 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:48.429848 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d8a28-69b2-4763-9525-218c30d7a0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:48.430835 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d8a28-69b2-4763-9525-218c30d7a0ce None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:48.431356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d8a28-69b2-4763-9525-218c30d7a0ce None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:48.431834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d8a28-69b2-4763-9525-218c30d7a0ce None admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:48.432824 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d8a28-69b2-4763-9525-218c30d7a0ce None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:48.433357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d8a28-69b2-4763-9525-218c30d7a0ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:48.435044 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d8a28-69b2-4763-9525-218c30d7a0ce None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:48.445689 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef632dd-ef04-449a-ba06-084c715bad15 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:48.454572 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 339/660] 10.222.0.40 () {56 vars in 1139 bytes} [Wed Aug 3 04:54:48 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:48.455674 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d8a28-69b2-4763-9525-218c30d7a0ce None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:48.481025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6fa75c-08b4-4fb9-a8b3-aaef917d115b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:48.481670 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fa75c-08b4-4fb9-a8b3-aaef917d115b None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:48.481960 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fa75c-08b4-4fb9-a8b3-aaef917d115b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:48.482238 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fa75c-08b4-4fb9-a8b3-aaef917d115b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:48.490739 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6fa75c-08b4-4fb9-a8b3-aaef917d115b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:48.492224 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6fa75c-08b4-4fb9-a8b3-aaef917d115b None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:48.504299 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-5e6fa75c-08b4-4fb9-a8b3-aaef917d115b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 03 04:54:48.504299 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:48.504299 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:48.504299 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:48.504299 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:48.504299 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:48.504299 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:48.504299 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:48.504299 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:48.504299 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:48.504299 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:48.504299 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:48.504299 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:54:48.504299 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:54:48.504299 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:54:48.504299 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:54:48.505274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:48.505274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:48.505274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:54:48.505274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:48.505274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:54:48.505274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:48.505274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:48.505274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:48.505274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:48.505274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:48.505274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:48.505274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:48.505274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:48.505274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:48.505274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:48.505274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:48.505274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:48.505274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:48.506306 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:48.506306 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:48.506306 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:54:48.506306 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:54:48.506306 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:54:48.506306 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:54:48.506306 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:54:48.506306 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:54:48.506306 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 03 04:54:48.506306 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:48.506306 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 322/661] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:54:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:48.506306 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6fa75c-08b4-4fb9-a8b3-aaef917d115b None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:48.548992 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:48.578653 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d8a28-69b2-4763-9525-218c30d7a0ce None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:48.611517 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:48.614775 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6fa75c-08b4-4fb9-a8b3-aaef917d115b None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:48.622150 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30964ec-d553-435c-ba08-2c6519a3e2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:48.623351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30964ec-d553-435c-ba08-2c6519a3e2d8 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:48.623975 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30964ec-d553-435c-ba08-2c6519a3e2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:48.624420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30964ec-d553-435c-ba08-2c6519a3e2d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:48.631324 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30964ec-d553-435c-ba08-2c6519a3e2d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:48.631324 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30964ec-d553-435c-ba08-2c6519a3e2d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:48.631324 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30964ec-d553-435c-ba08-2c6519a3e2d8 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:48.641832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 340/662] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:54:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:48.643046 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30964ec-d553-435c-ba08-2c6519a3e2d8 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:48.658207 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63436a5-1e84-48e4-a98e-e35a7bc74f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:48.658954 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63436a5-1e84-48e4-a98e-e35a7bc74f69 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:48.659240 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63436a5-1e84-48e4-a98e-e35a7bc74f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:48.659521 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63436a5-1e84-48e4-a98e-e35a7bc74f69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:48.734927 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63436a5-1e84-48e4-a98e-e35a7bc74f69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:48.736407 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63436a5-1e84-48e4-a98e-e35a7bc74f69 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:48.762284 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30964ec-d553-435c-ba08-2c6519a3e2d8 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:48.776433 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:48.801646 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b2761-2271-418b-811e-4d95e7139e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:48.802634 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b2761-2271-418b-811e-4d95e7139e06 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:48.803152 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b2761-2271-418b-811e-4d95e7139e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:48.803633 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b2761-2271-418b-811e-4d95e7139e06 None admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:48.804661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b2761-2271-418b-811e-4d95e7139e06 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:48.805175 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b2761-2271-418b-811e-4d95e7139e06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:48.806918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b2761-2271-418b-811e-4d95e7139e06 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:48.827071 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 341/663] 10.222.0.40 () {56 vars in 1137 bytes} [Wed Aug 3 04:54:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:48.828648 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b2761-2271-418b-811e-4d95e7139e06 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:48.852445 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 323/664] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:54:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:54:48.853440 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63436a5-1e84-48e4-a98e-e35a7bc74f69 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:48.880211 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:48.941794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b2761-2271-418b-811e-4d95e7139e06 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:48.957869 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63436a5-1e84-48e4-a98e-e35a7bc74f69 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:48.979135 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50abbd2a-b2cf-4405-ba80-b0169f800d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:48.979802 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50abbd2a-b2cf-4405-ba80-b0169f800d40 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:48.980089 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50abbd2a-b2cf-4405-ba80-b0169f800d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:48.980360 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50abbd2a-b2cf-4405-ba80-b0169f800d40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:48.981564 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50abbd2a-b2cf-4405-ba80-b0169f800d40 None admin] RBAC: Adding query filter params (role.id=cae571342db14f5fac7849efe7f8636e, scope.project.id=cf3763a37383456184a9ca9351b4ce03, user.id=a3fa4a6c16274c18849869b55b33ac62) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:48.982076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50abbd2a-b2cf-4405-ba80-b0169f800d40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:48.983573 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50abbd2a-b2cf-4405-ba80-b0169f800d40 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:48.998870 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 342/665] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:54:48 2022] GET /identity/v3/role_assignments?user.id=a3fa4a6c16274c18849869b55b33ac62&scope.project.id=cf3763a37383456184a9ca9351b4ce03&role.id=cae571342db14f5fac7849efe7f8636e => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:48.998870 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50abbd2a-b2cf-4405-ba80-b0169f800d40 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:49.005222 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69217d6-015e-4946-93b5-bcc71107f8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:49.006002 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69217d6-015e-4946-93b5-bcc71107f8fc None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:49.006293 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69217d6-015e-4946-93b5-bcc71107f8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:49.006577 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69217d6-015e-4946-93b5-bcc71107f8fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:49.017620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:49.095768 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69217d6-015e-4946-93b5-bcc71107f8fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:49.097599 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69217d6-015e-4946-93b5-bcc71107f8fc None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:49.147703 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50abbd2a-b2cf-4405-ba80-b0169f800d40 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:49.161211 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:49.198359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627ba65-5e1d-4639-9f0a-7963be930877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:49.199079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627ba65-5e1d-4639-9f0a-7963be930877 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:49.199368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627ba65-5e1d-4639-9f0a-7963be930877 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:49.199648 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627ba65-5e1d-4639-9f0a-7963be930877 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:49.213172 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d627ba65-5e1d-4639-9f0a-7963be930877 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:49.214648 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d627ba65-5e1d-4639-9f0a-7963be930877 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:49.220519 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 324/666] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:54:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:54:49.221634 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244261e9-386c-42cf-a3c5-60882493cc5d None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:49.221904 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244261e9-386c-42cf-a3c5-60882493cc5d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:49.222182 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244261e9-386c-42cf-a3c5-60882493cc5d None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:49.223790 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 325/667] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:49.230692 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-d627ba65-5e1d-4639-9f0a-7963be930877 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:49.230692 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:49.230692 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:49.230692 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:49.230692 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:49.230692 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:49.230692 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:49.230692 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:49.230692 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:49.230692 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:49.230692 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:49.230692 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:49.230692 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:49.230692 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:49.230692 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:49.231813 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:49.231813 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:49.231813 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:49.231813 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:49.231813 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:49.231813 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:49.231813 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:49.231813 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:49.231813 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:49.231813 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:49.231813 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:49.231813 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:49.231813 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:49.231813 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:49.231813 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:49.231813 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:49.231813 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:49.232942 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:49.232942 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:49.232942 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:49.232942 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:49.233755 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd6eb9-1ed5-4e12-9b79-5811f9adf6a9 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:49.233755 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd6eb9-1ed5-4e12-9b79-5811f9adf6a9 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:49.233960 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd6eb9-1ed5-4e12-9b79-5811f9adf6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:49.235464 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 343/668] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:49.236271 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627ba65-5e1d-4639-9f0a-7963be930877 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:49.255228 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-19fd6eb9-1ed5-4e12-9b79-5811f9adf6a9 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:49.256424 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-19fd6eb9-1ed5-4e12-9b79-5811f9adf6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:49.297951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:49.350298 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 326/669] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:49.351256 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd6eb9-1ed5-4e12-9b79-5811f9adf6a9 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:49.360643 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627ba65-5e1d-4639-9f0a-7963be930877 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:49.397324 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ba18c-6127-43cb-b2fd-f0d773b5b36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:49.397973 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ba18c-6127-43cb-b2fd-f0d773b5b36b None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:49.398255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ba18c-6127-43cb-b2fd-f0d773b5b36b None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:49.398793 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ba18c-6127-43cb-b2fd-f0d773b5b36b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:49.410503 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2ba18c-6127-43cb-b2fd-f0d773b5b36b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:49.412176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2ba18c-6127-43cb-b2fd-f0d773b5b36b None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:49.427918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-ea2ba18c-6127-43cb-b2fd-f0d773b5b36b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:49.427918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:49.427918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:49.427918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:49.427918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:49.427918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:49.427918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:49.427918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:49.427918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:49.427918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:49.427918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:49.427918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:49.427918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:49.427918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:49.427918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:49.428846 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:49.428846 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:49.428846 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:49.428846 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:49.428846 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:49.428846 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:49.428846 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:49.428846 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:49.428846 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:49.428846 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:49.428846 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:49.428846 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:49.428846 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:49.428846 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:49.428846 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:49.428846 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:49.428846 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:49.429748 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:49.429748 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:49.429748 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:49.429748 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:49.433611 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 344/670] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:49.434551 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ba18c-6127-43cb-b2fd-f0d773b5b36b None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:49.450443 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:49.489698 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd6eb9-1ed5-4e12-9b79-5811f9adf6a9 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:49.537934 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580a06c-a84b-424a-9058-ee2909a3f619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:49.539592 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580a06c-a84b-424a-9058-ee2909a3f619 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:49.540060 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580a06c-a84b-424a-9058-ee2909a3f619 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:49.540538 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580a06c-a84b-424a-9058-ee2909a3f619 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:49.541383 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ba18c-6127-43cb-b2fd-f0d773b5b36b None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:49.541588 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9580a06c-a84b-424a-9058-ee2909a3f619 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:49.542072 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9580a06c-a84b-424a-9058-ee2909a3f619 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:49.544790 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9580a06c-a84b-424a-9058-ee2909a3f619 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:49.567019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 327/671] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:49.568460 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580a06c-a84b-424a-9058-ee2909a3f619 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:49.581534 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15ddbb-a00a-4bf7-9814-04dca8136658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:49.582224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15ddbb-a00a-4bf7-9814-04dca8136658 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:49.582501 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15ddbb-a00a-4bf7-9814-04dca8136658 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:49.582773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15ddbb-a00a-4bf7-9814-04dca8136658 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:49.594446 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab15ddbb-a00a-4bf7-9814-04dca8136658 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:49.599877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-ab15ddbb-a00a-4bf7-9814-04dca8136658 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:49.599877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:49.599877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:49.599877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:49.599877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:49.599877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:49.599877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:49.599877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:49.599877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:49.599877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:49.599877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:49.599877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:49.599877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:49.599877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:49.599877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:49.599877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:49.600856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:49.600856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:49.600856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:49.600856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:49.600856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:49.600856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:49.600856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:49.600856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:49.600856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:49.600856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:49.600856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:49.600856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:49.600856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:49.600856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:49.600856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:49.600856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:49.601738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:49.601738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:49.601738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:49.601738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:49.601738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:49.601738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:49.601738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:49.601738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:49.601738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:49.601738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:49.601738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:49.601738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:49.601738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 345/672] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 3 04:54:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:49.602525 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15ddbb-a00a-4bf7-9814-04dca8136658 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:49.611928 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:49.693244 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:49.696377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580a06c-a84b-424a-9058-ee2909a3f619 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:49.731190 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15ddbb-a00a-4bf7-9814-04dca8136658 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:49.732735 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905db6e-5d2e-4d81-aa84-85d9494e0c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:49.733354 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905db6e-5d2e-4d81-aa84-85d9494e0c3d None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:49.733635 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905db6e-5d2e-4d81-aa84-85d9494e0c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:49.733917 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905db6e-5d2e-4d81-aa84-85d9494e0c3d None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:49.734606 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905db6e-5d2e-4d81-aa84-85d9494e0c3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:49.735043 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905db6e-5d2e-4d81-aa84-85d9494e0c3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:49.736430 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905db6e-5d2e-4d81-aa84-85d9494e0c3d None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:49.749882 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 328/673] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:49.750537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905db6e-5d2e-4d81-aa84-85d9494e0c3d None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:49.775110 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f5dc8-7c3a-4750-86ec-352b053fb93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:49.775763 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f5dc8-7c3a-4750-86ec-352b053fb93b None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:49.776049 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f5dc8-7c3a-4750-86ec-352b053fb93b None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:49.776325 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f5dc8-7c3a-4750-86ec-352b053fb93b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:49.784422 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f5dc8-7c3a-4750-86ec-352b053fb93b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:49.785873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f5dc8-7c3a-4750-86ec-352b053fb93b None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:49.798163 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-cf3f5dc8-7c3a-4750-86ec-352b053fb93b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:49.798163 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:49.798163 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:49.798163 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:49.798163 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:49.798163 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:49.798163 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:49.798163 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:49.798163 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:49.798163 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:49.798163 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:49.798163 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:49.798163 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:54:49.798163 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:54:49.798163 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:54:49.799381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:54:49.799381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:49.799381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:49.799381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:49.799381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:49.799381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:49.799381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:49.799381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:49.799381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:49.799381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:49.799381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:49.799381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:49.799381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:49.799381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:49.799381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:49.799381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:49.799381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:49.805033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:54:49.805033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:54:49.805033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:54:49.805033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:54:49.805033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:54:49.805033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:54:49.805033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:49.805033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:49.805033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 346/674] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:54:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:49.805033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8dfa49-0f2c-46f6-938f-d1de58db2cf0 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:49.805033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8dfa49-0f2c-46f6-938f-d1de58db2cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:49.805642 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8dfa49-0f2c-46f6-938f-d1de58db2cf0 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:49.805642 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 347/675] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:49.805642 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8dfa49-0f2c-46f6-938f-d1de58db2cf0 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:49.854212 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:49.866334 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905db6e-5d2e-4d81-aa84-85d9494e0c3d None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:49.902245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959804f-1739-477a-b49c-756e2dcaba11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:49.902880 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959804f-1739-477a-b49c-756e2dcaba11 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:49.903174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959804f-1739-477a-b49c-756e2dcaba11 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:49.903451 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959804f-1739-477a-b49c-756e2dcaba11 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:49.904172 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b959804f-1739-477a-b49c-756e2dcaba11 None admin] RBAC: Adding query filter params (name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:49.904475 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b959804f-1739-477a-b49c-756e2dcaba11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:49.906476 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b959804f-1739-477a-b49c-756e2dcaba11 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:49.915379 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 329/676] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 3 04:54:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:49.916199 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959804f-1739-477a-b49c-756e2dcaba11 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:49.927491 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8dfa49-0f2c-46f6-938f-d1de58db2cf0 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:49.962759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:49.969115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadd4e0a-39ec-47e3-8e01-45af9945f31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:49.969883 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd4e0a-39ec-47e3-8e01-45af9945f31c None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:49.975115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd4e0a-39ec-47e3-8e01-45af9945f31c None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:49.975115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd4e0a-39ec-47e3-8e01-45af9945f31c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:49.984118 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadd4e0a-39ec-47e3-8e01-45af9945f31c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:49.985014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadd4e0a-39ec-47e3-8e01-45af9945f31c None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:49.999231 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-cadd4e0a-39ec-47e3-8e01-45af9945f31c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:49.999231 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:49.999231 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:49.999231 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:49.999231 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:49.999231 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:49.999231 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:49.999231 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:49.999231 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:49.999231 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:49.999231 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:49.999231 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:49.999231 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:54:49.999231 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:54:49.999231 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:54:50.000162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:54:50.000162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:50.000162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:50.000162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:50.000162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:50.000162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:50.000162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:50.000162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:50.000162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:50.000162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:50.000162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:50.000162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:50.000162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:50.000162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:50.000162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:50.000162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:50.000162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:50.001058 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:54:50.001058 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:54:50.001058 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:54:50.001058 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:54:50.001058 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:54:50.001058 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:54:50.001058 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:50.001058 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:50.005296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 348/677] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:54:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:50.006668 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5521ed-ec0d-4c97-a45a-a4088ed51ece None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:50.006991 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5521ed-ec0d-4c97-a45a-a4088ed51ece None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:50.007279 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5521ed-ec0d-4c97-a45a-a4088ed51ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:50.028400 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-8d5521ed-ec0d-4c97-a45a-a4088ed51ece None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:50.029038 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959804f-1739-477a-b49c-756e2dcaba11 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:50.029605 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-8d5521ed-ec0d-4c97-a45a-a4088ed51ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:50.064294 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b9a97-3cf3-4241-9e26-87706b695536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:50.064903 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b9a97-3cf3-4241-9e26-87706b695536 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:50.065201 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b9a97-3cf3-4241-9e26-87706b695536 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:50.065454 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b9a97-3cf3-4241-9e26-87706b695536 None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:50.066186 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b9a97-3cf3-4241-9e26-87706b695536 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:50.066489 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b9a97-3cf3-4241-9e26-87706b695536 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:50.067930 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b9a97-3cf3-4241-9e26-87706b695536 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:50.080653 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 330/678] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:54:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:50.081441 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b9a97-3cf3-4241-9e26-87706b695536 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:50.096785 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:50.108958 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 349/679] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:50.108958 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5521ed-ec0d-4c97-a45a-a4088ed51ece None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:50.209453 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b9a97-3cf3-4241-9e26-87706b695536 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:50.213845 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5521ed-ec0d-4c97-a45a-a4088ed51ece None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:50.237819 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:50.249421 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406b39f-ba9a-496e-8927-18c4baf1308d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:50.249735 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e040a-0beb-4c26-9f76-bdf30b950657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:50.251879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e040a-0beb-4c26-9f76-bdf30b950657 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:50.251987 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406b39f-ba9a-496e-8927-18c4baf1308d None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:50.252226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e040a-0beb-4c26-9f76-bdf30b950657 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:50.252339 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406b39f-ba9a-496e-8927-18c4baf1308d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:50.252416 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e040a-0beb-4c26-9f76-bdf30b950657 None admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:50.252501 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406b39f-ba9a-496e-8927-18c4baf1308d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:50.252577 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e040a-0beb-4c26-9f76-bdf30b950657 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:50.252577 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e040a-0beb-4c26-9f76-bdf30b950657 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:50.253474 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e040a-0beb-4c26-9f76-bdf30b950657 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:50.263033 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2406b39f-ba9a-496e-8927-18c4baf1308d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:50.265141 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2406b39f-ba9a-496e-8927-18c4baf1308d None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:50.270371 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 350/680] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:54:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:50.271379 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e040a-0beb-4c26-9f76-bdf30b950657 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:50.282134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-2406b39f-ba9a-496e-8927-18c4baf1308d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:50.282134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:50.282134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:50.282134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:50.282134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:50.282134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:50.282134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:50.282134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:50.282134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:50.282134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:50.282134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:50.282134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:50.282134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:50.282134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:50.282134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:50.283105 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:50.283105 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:50.283105 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:50.283105 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:50.283105 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:50.283105 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:50.283105 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:50.283105 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:50.283105 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:50.283105 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:50.283105 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:50.283105 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:50.283105 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:50.283105 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:50.283105 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:50.283105 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:50.283105 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:50.284005 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:50.284005 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:50.284005 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:50.284005 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:50.284005 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 331/681] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:50.284301 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406b39f-ba9a-496e-8927-18c4baf1308d None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:50.354099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:50.383300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e040a-0beb-4c26-9f76-bdf30b950657 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:50.396347 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406b39f-ba9a-496e-8927-18c4baf1308d None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:50.419534 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0bd230-6b4a-4df1-a13c-18b60fa4eba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:50.420179 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0bd230-6b4a-4df1-a13c-18b60fa4eba5 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:50.420459 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0bd230-6b4a-4df1-a13c-18b60fa4eba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:50.421276 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0bd230-6b4a-4df1-a13c-18b60fa4eba5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:50.422125 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0bd230-6b4a-4df1-a13c-18b60fa4eba5 None admin] RBAC: Adding query filter params (role.id=cae571342db14f5fac7849efe7f8636e, scope.project.id=cf3763a37383456184a9ca9351b4ce03, user.id=cff15ea4cbc64b068aa5f6586b5bff87) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:50.422432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0bd230-6b4a-4df1-a13c-18b60fa4eba5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:50.423906 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0bd230-6b4a-4df1-a13c-18b60fa4eba5 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:50.429661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 351/682] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:54:50 2022] GET /identity/v3/role_assignments?user.id=cff15ea4cbc64b068aa5f6586b5bff87&scope.project.id=cf3763a37383456184a9ca9351b4ce03&role.id=cae571342db14f5fac7849efe7f8636e => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:50.430431 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0bd230-6b4a-4df1-a13c-18b60fa4eba5 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:50.441965 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c0676-7fd6-4de9-8f72-2b602e87cb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:50.443085 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c0676-7fd6-4de9-8f72-2b602e87cb5a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:50.443564 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c0676-7fd6-4de9-8f72-2b602e87cb5a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:50.444030 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c0676-7fd6-4de9-8f72-2b602e87cb5a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:50.444873 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97c0676-7fd6-4de9-8f72-2b602e87cb5a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:50.446550 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97c0676-7fd6-4de9-8f72-2b602e87cb5a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:50.466419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-b97c0676-7fd6-4de9-8f72-2b602e87cb5a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 04:54:50.466419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:50.466419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 04:54:50.466419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 04:54:50.466419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 04:54:50.466419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 04:54:50.466419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 04:54:50.466419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:54:50.466419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 04:54:50.466419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:50.466419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:54:50.466419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:50.466419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:50.466419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:50.466419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:50.466419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:50.467643 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:50.467643 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:50.467643 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:50.467643 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:50.467643 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:50.467643 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:50.467643 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:50.467643 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 04:54:50.467643 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 04:54:50.467643 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 04:54:50.467643 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 04:54:50.467643 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:50.467643 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:50.467643 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:50.467643 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:50.467643 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:50.468493 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:50.468493 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:50.468493 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:50.468493 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:50.468493 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:50.468493 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:50.468493 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:50.468493 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:50.468493 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:50.468493 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:50.468493 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:50.468493 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 04:54:50.468493 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:54:50.468493 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 04:54:50.468493 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:50.476199 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 332/683] 10.222.0.40 () {56 vars in 1129 bytes} [Wed Aug 3 04:54:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:50.477424 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c0676-7fd6-4de9-8f72-2b602e87cb5a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:50.490508 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:50.549883 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0bd230-6b4a-4df1-a13c-18b60fa4eba5 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:50.594413 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c192220-8d30-4b31-82f7-ea23b39791e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:50.595469 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c192220-8d30-4b31-82f7-ea23b39791e6 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:50.595823 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c192220-8d30-4b31-82f7-ea23b39791e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:50.596166 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c192220-8d30-4b31-82f7-ea23b39791e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:50.598649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c192220-8d30-4b31-82f7-ea23b39791e6 None admin] RBAC: Adding query filter params (role.id=cae571342db14f5fac7849efe7f8636e, scope.project.id=cf3763a37383456184a9ca9351b4ce03, user.id=84cc14b33895443b893fab487bfe1858) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:50.598649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c192220-8d30-4b31-82f7-ea23b39791e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:50.600566 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c192220-8d30-4b31-82f7-ea23b39791e6 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:50.600680 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c0676-7fd6-4de9-8f72-2b602e87cb5a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:50.607545 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 352/684] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:54:50 2022] GET /identity/v3/role_assignments?user.id=84cc14b33895443b893fab487bfe1858&scope.project.id=cf3763a37383456184a9ca9351b4ce03&role.id=cae571342db14f5fac7849efe7f8636e => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:50.608732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c192220-8d30-4b31-82f7-ea23b39791e6 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:50.636292 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:50.646972 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f2f9d-e360-4f0f-a591-1ca397cc4e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:50.648067 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f2f9d-e360-4f0f-a591-1ca397cc4e89 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:50.648788 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f2f9d-e360-4f0f-a591-1ca397cc4e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:50.653075 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f2f9d-e360-4f0f-a591-1ca397cc4e89 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:50.653075 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95f2f9d-e360-4f0f-a591-1ca397cc4e89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:50.653075 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95f2f9d-e360-4f0f-a591-1ca397cc4e89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:50.653075 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95f2f9d-e360-4f0f-a591-1ca397cc4e89 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:50.674704 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 333/685] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:50.680225 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f2f9d-e360-4f0f-a591-1ca397cc4e89 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:50.755288 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c192220-8d30-4b31-82f7-ea23b39791e6 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:50.770496 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:50.808015 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f18125c-986b-403c-bd64-d8e0ecd49793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:50.808676 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18125c-986b-403c-bd64-d8e0ecd49793 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:50.808958 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18125c-986b-403c-bd64-d8e0ecd49793 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:50.809235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18125c-986b-403c-bd64-d8e0ecd49793 None admin] PATH_INFO: `/v3/services` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:50.810286 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f18125c-986b-403c-bd64-d8e0ecd49793 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:50.810598 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f18125c-986b-403c-bd64-d8e0ecd49793 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:50.812692 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f18125c-986b-403c-bd64-d8e0ecd49793 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:50.819218 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 353/686] 10.222.0.40 () {56 vars in 1124 bytes} [Wed Aug 3 04:54:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:50.824599 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f18125c-986b-403c-bd64-d8e0ecd49793 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:50.837028 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f2f9d-e360-4f0f-a591-1ca397cc4e89 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:50.881472 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530458a7-faf5-4c45-862a-bb09a6b595f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:50.882127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530458a7-faf5-4c45-862a-bb09a6b595f4 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:50.882461 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530458a7-faf5-4c45-862a-bb09a6b595f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:50.882736 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530458a7-faf5-4c45-862a-bb09a6b595f4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:50.891908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530458a7-faf5-4c45-862a-bb09a6b595f4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:50.893349 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530458a7-faf5-4c45-862a-bb09a6b595f4 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:50.908765 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-530458a7-faf5-4c45-862a-bb09a6b595f4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 03 04:54:50.908765 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:50.908765 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:50.908765 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:50.908765 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:50.908765 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:50.908765 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:50.908765 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:50.908765 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:50.908765 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:50.908765 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:50.908765 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:50.908765 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:54:50.908765 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:54:50.908765 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:54:50.908765 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:54:50.909771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:50.909771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:50.909771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:54:50.909771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:50.909771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:54:50.909771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:50.909771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:50.909771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:50.909771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:50.909771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:50.909771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:50.909771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:50.909771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:50.909771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:50.909771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:50.909771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:50.909771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:50.909771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:50.910707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:50.910707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:50.910707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:54:50.910707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:54:50.910707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:54:50.910707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:54:50.910707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:54:50.910707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:54:50.910707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 03 04:54:50.910707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:50.914683 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 334/687] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:54:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:50.916037 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530458a7-faf5-4c45-862a-bb09a6b595f4 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:50.965722 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f18125c-986b-403c-bd64-d8e0ecd49793 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:50.978156 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:51.014060 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4d77ba-500b-45bc-b196-e4080badd340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:51.015548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d77ba-500b-45bc-b196-e4080badd340 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:51.016449 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d77ba-500b-45bc-b196-e4080badd340 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:51.017088 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d77ba-500b-45bc-b196-e4080badd340 None admin] PATH_INFO: `/v3/services` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:51.018093 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4d77ba-500b-45bc-b196-e4080badd340 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:51.018614 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4d77ba-500b-45bc-b196-e4080badd340 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:51.020787 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4d77ba-500b-45bc-b196-e4080badd340 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:51.029116 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 354/688] 10.222.0.40 () {56 vars in 1124 bytes} [Wed Aug 3 04:54:50 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:51.047814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530458a7-faf5-4c45-862a-bb09a6b595f4 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:51.085412 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0599d5-b642-4a0f-9e82-511d440a2fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:51.086104 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0599d5-b642-4a0f-9e82-511d440a2fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:51.086390 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0599d5-b642-4a0f-9e82-511d440a2fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:51.086664 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0599d5-b642-4a0f-9e82-511d440a2fd1 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:51.087553 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0599d5-b642-4a0f-9e82-511d440a2fd1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:51.087862 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0599d5-b642-4a0f-9e82-511d440a2fd1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:51.089315 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0599d5-b642-4a0f-9e82-511d440a2fd1 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:51.114995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 335/689] 10.222.0.40 () {56 vars in 1135 bytes} [Wed Aug 3 04:54:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:51.117014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4d77ba-500b-45bc-b196-e4080badd340 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:51.170362 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:51.242502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4d77ba-500b-45bc-b196-e4080badd340 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:51.283413 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f34592-441f-48fe-a7b3-00bd7634249e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:51.284040 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f34592-441f-48fe-a7b3-00bd7634249e None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:51.284319 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f34592-441f-48fe-a7b3-00bd7634249e None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:51.284591 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f34592-441f-48fe-a7b3-00bd7634249e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:51.309041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5eff1-ef06-41a6-b9cb-74226b5e27a0 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:51.317392 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5eff1-ef06-41a6-b9cb-74226b5e27a0 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:51.317392 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5eff1-ef06-41a6-b9cb-74226b5e27a0 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:51.317392 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 336/690] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:54:51.317392 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560ede3-ddbc-4800-b31d-95e34c6428a1 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:51.317392 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560ede3-ddbc-4800-b31d-95e34c6428a1 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:51.317392 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560ede3-ddbc-4800-b31d-95e34c6428a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:51.339588 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-2560ede3-ddbc-4800-b31d-95e34c6428a1 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:51.340808 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2560ede3-ddbc-4800-b31d-95e34c6428a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:51.363327 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f34592-441f-48fe-a7b3-00bd7634249e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:51.365252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f34592-441f-48fe-a7b3-00bd7634249e None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:51.415221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:51.443517 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 337/691] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 04:54:51.466457 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560ede3-ddbc-4800-b31d-95e34c6428a1 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:51.531817 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 355/692] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:54:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:54:51.532162 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:51.535389 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f34592-441f-48fe-a7b3-00bd7634249e None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:51.656705 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560ede3-ddbc-4800-b31d-95e34c6428a1 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:51.674834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f34592-441f-48fe-a7b3-00bd7634249e None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:51.728662 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7a435-2ff7-40cc-bf39-b39ccb0916ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:51.729230 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7a435-2ff7-40cc-bf39-b39ccb0916ea None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:51.729507 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7a435-2ff7-40cc-bf39-b39ccb0916ea None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:51.729776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7a435-2ff7-40cc-bf39-b39ccb0916ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:51.742668 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bbb5af-e96c-44ab-aac1-97c3b08479bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:51.743437 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bbb5af-e96c-44ab-aac1-97c3b08479bb None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:51.743720 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bbb5af-e96c-44ab-aac1-97c3b08479bb None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:51.743992 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bbb5af-e96c-44ab-aac1-97c3b08479bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:51.754895 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7a435-2ff7-40cc-bf39-b39ccb0916ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:51.759888 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-f9a7a435-2ff7-40cc-bf39-b39ccb0916ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:54:51.759888 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:51.759888 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:51.759888 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:51.759888 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:51.759888 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:51.759888 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:51.759888 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:51.759888 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:51.759888 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:51.759888 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:51.759888 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:51.759888 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:51.759888 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:51.759888 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:51.759888 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:51.760897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:51.760897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:51.760897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:51.760897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:51.760897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:51.760897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:51.760897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:51.760897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:51.760897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:51.760897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:51.760897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:51.760897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:51.760897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:51.760897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:51.760897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:51.760897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:51.761749 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:51.761749 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:51.761749 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:51.761749 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:51.761749 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:51.761749 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:51.761749 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:51.761749 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:51.761749 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:51.761749 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:51.761749 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:54:51.761749 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:51.763307 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:51.763727 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 338/693] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:54:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:54:51.766540 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bbb5af-e96c-44ab-aac1-97c3b08479bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:51.768177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bbb5af-e96c-44ab-aac1-97c3b08479bb None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:51.770205 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7a435-2ff7-40cc-bf39-b39ccb0916ea None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:51.796502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-d1bbb5af-e96c-44ab-aac1-97c3b08479bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:51.796502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:51.796502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:51.796502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:51.796502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:51.796502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:51.796502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:51.796502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:51.796502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:51.796502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:51.796502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:51.796502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:51.796502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:51.796502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:51.796502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:51.798642 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:51.798642 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:51.798642 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:51.798642 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:51.798642 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:51.798642 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:51.798642 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:51.798642 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:51.798642 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:51.798642 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:51.798642 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:51.798642 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:51.798642 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:51.798642 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:51.798642 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:51.798642 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:51.798642 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:51.800826 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:51.800826 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:51.800826 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:51.800826 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:51.807184 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 356/694] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:51.808393 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bbb5af-e96c-44ab-aac1-97c3b08479bb None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:51.954697 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:51.961972 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7a435-2ff7-40cc-bf39-b39ccb0916ea None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:52.016354 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f809a-1eef-4b21-8303-9c3a7fa7c65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:52.017178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f809a-1eef-4b21-8303-9c3a7fa7c65e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:52.017465 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f809a-1eef-4b21-8303-9c3a7fa7c65e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:52.017742 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f809a-1eef-4b21-8303-9c3a7fa7c65e None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:52.018467 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f809a-1eef-4b21-8303-9c3a7fa7c65e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:52.018774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f809a-1eef-4b21-8303-9c3a7fa7c65e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:52.022049 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f809a-1eef-4b21-8303-9c3a7fa7c65e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:52.025478 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bbb5af-e96c-44ab-aac1-97c3b08479bb None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:52.040480 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 339/695] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:54:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:52.047979 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f809a-1eef-4b21-8303-9c3a7fa7c65e None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:52.078117 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b384c-fb6f-47ca-a224-4d97fac1243e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:52.078746 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b384c-fb6f-47ca-a224-4d97fac1243e None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:52.079071 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b384c-fb6f-47ca-a224-4d97fac1243e None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:52.079349 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b384c-fb6f-47ca-a224-4d97fac1243e None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:52.080048 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b384c-fb6f-47ca-a224-4d97fac1243e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:52.081528 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b384c-fb6f-47ca-a224-4d97fac1243e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:52.082957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b384c-fb6f-47ca-a224-4d97fac1243e None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:52.111292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 357/696] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:52.114683 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b384c-fb6f-47ca-a224-4d97fac1243e None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:52.183189 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:52.239978 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f809a-1eef-4b21-8303-9c3a7fa7c65e None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:52.280761 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2137f2-b8bc-4641-adca-67616cfb0ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:52.281437 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2137f2-b8bc-4641-adca-67616cfb0ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:52.281719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2137f2-b8bc-4641-adca-67616cfb0ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:52.281988 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2137f2-b8bc-4641-adca-67616cfb0ac1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:52.293691 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2137f2-b8bc-4641-adca-67616cfb0ac1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:52.295313 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2137f2-b8bc-4641-adca-67616cfb0ac1 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:52.311863 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b384c-fb6f-47ca-a224-4d97fac1243e None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:52.312054 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-1d2137f2-b8bc-4641-adca-67616cfb0ac1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:52.312054 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:52.312054 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:52.312054 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:52.312054 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:52.312054 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:52.312054 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:52.312054 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:52.312054 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:52.312054 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:52.312054 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:52.312054 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:52.312054 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:52.312054 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:52.312054 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:52.312908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:52.312908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:52.312908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:52.312908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:52.312908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:52.312908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:52.312908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:52.312908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:52.312908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:52.312908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:52.312908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:52.312908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:52.312908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:52.312908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:52.312908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:52.312908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:52.312908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:52.314147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:52.314147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:52.314147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:52.314147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:52.314147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 340/697] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:52.320190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2137f2-b8bc-4641-adca-67616cfb0ac1 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:52.338498 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:52.408607 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc48037-1b5b-4871-a5e9-23cafea81506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:52.409276 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc48037-1b5b-4871-a5e9-23cafea81506 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:52.409560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc48037-1b5b-4871-a5e9-23cafea81506 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:52.409832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc48037-1b5b-4871-a5e9-23cafea81506 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:52.427953 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc48037-1b5b-4871-a5e9-23cafea81506 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:52.429501 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc48037-1b5b-4871-a5e9-23cafea81506 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:52.444163 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2137f2-b8bc-4641-adca-67616cfb0ac1 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:52.456745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-bbc48037-1b5b-4871-a5e9-23cafea81506 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:52.456745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:52.456745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:52.456745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:52.456745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:52.456745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:52.456745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:52.456745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:52.456745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:52.456745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:52.456745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:52.456745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:52.456745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:54:52.456745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:54:52.456745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:54:52.457720 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:54:52.457720 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:52.457720 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:52.457720 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:52.457720 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:52.457720 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:52.457720 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:52.457720 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:52.457720 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:52.457720 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:52.457720 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:52.457720 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:52.457720 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:52.457720 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:52.457720 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:52.457720 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:52.457720 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:52.458661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:54:52.458661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:54:52.458661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:54:52.458661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:54:52.458661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:54:52.458661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:54:52.458661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:52.458661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:52.462573 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 358/698] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:54:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:52.463776 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc48037-1b5b-4871-a5e9-23cafea81506 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:52.502244 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163b430-aad8-4dd6-9aec-410a09252217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:52.506326 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163b430-aad8-4dd6-9aec-410a09252217 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:52.506904 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163b430-aad8-4dd6-9aec-410a09252217 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:52.507507 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163b430-aad8-4dd6-9aec-410a09252217 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:52.508447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163b430-aad8-4dd6-9aec-410a09252217 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:52.508960 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163b430-aad8-4dd6-9aec-410a09252217 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:52.512495 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163b430-aad8-4dd6-9aec-410a09252217 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:52.538816 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 341/699] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:52.563911 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163b430-aad8-4dd6-9aec-410a09252217 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:52.609559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc48037-1b5b-4871-a5e9-23cafea81506 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:52.621579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:52.646668 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec396b-61ae-44b7-a788-1524bf41f994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:52.647690 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec396b-61ae-44b7-a788-1524bf41f994 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:52.648227 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec396b-61ae-44b7-a788-1524bf41f994 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:52.648992 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec396b-61ae-44b7-a788-1524bf41f994 None admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:52.649946 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfec396b-61ae-44b7-a788-1524bf41f994 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:52.650464 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfec396b-61ae-44b7-a788-1524bf41f994 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:52.652365 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfec396b-61ae-44b7-a788-1524bf41f994 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:52.666177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 359/700] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:54:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:52.675659 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec396b-61ae-44b7-a788-1524bf41f994 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:52.707242 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:52.767278 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163b430-aad8-4dd6-9aec-410a09252217 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:52.832944 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260da95d-e59f-4474-bfbe-6996a4711d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:52.833751 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260da95d-e59f-4474-bfbe-6996a4711d67 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:52.834036 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260da95d-e59f-4474-bfbe-6996a4711d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:52.834307 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260da95d-e59f-4474-bfbe-6996a4711d67 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:52.844734 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260da95d-e59f-4474-bfbe-6996a4711d67 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:52.846382 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260da95d-e59f-4474-bfbe-6996a4711d67 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:52.849091 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec396b-61ae-44b7-a788-1524bf41f994 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:52.859316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-260da95d-e59f-4474-bfbe-6996a4711d67 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 04:54:52.859316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:52.859316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:52.859316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:52.859316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:52.859316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:52.859316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:52.859316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:52.859316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:52.859316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:52.859316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:52.859316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:52.859316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:54:52.859316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:54:52.859316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:54:52.859316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:54:52.860286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:52.860286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:52.860286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:54:52.860286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:52.860286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:54:52.860286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:52.860286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:52.860286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:52.860286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:52.860286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:52.860286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:52.860286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:52.860286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:52.860286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:52.860286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:52.860286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:52.860286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:52.860286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:52.861320 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:52.861320 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:52.861320 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:54:52.861320 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:54:52.861320 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:54:52.861320 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:54:52.861320 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:54:52.861320 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:54:52.861320 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 04:54:52.861320 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:52.863998 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 342/701] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Aug 3 04:54:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:52.876122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260da95d-e59f-4474-bfbe-6996a4711d67 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:52.895111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b144d21-0696-4c4d-95bc-c83bbae257c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:52.896387 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b144d21-0696-4c4d-95bc-c83bbae257c0 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:52.896859 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b144d21-0696-4c4d-95bc-c83bbae257c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:52.897444 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b144d21-0696-4c4d-95bc-c83bbae257c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:52.898854 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b144d21-0696-4c4d-95bc-c83bbae257c0 None admin] RBAC: Adding query filter params (role.id=cae571342db14f5fac7849efe7f8636e, scope.project.id=cf3763a37383456184a9ca9351b4ce03, user.id=e1014bcb4d36441aa46042941c9a63d0) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:52.899352 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b144d21-0696-4c4d-95bc-c83bbae257c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:52.901173 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b144d21-0696-4c4d-95bc-c83bbae257c0 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:52.909494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 360/702] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:54:52 2022] GET /identity/v3/role_assignments?user.id=e1014bcb4d36441aa46042941c9a63d0&scope.project.id=cf3763a37383456184a9ca9351b4ce03&role.id=cae571342db14f5fac7849efe7f8636e => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:52.910720 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ded7ba-11da-4955-a7cf-e17939ad24ab None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:52.911039 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ded7ba-11da-4955-a7cf-e17939ad24ab None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:52.911317 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ded7ba-11da-4955-a7cf-e17939ad24ab None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:52.917282 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 361/703] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:52.919344 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf010c9-08ab-412c-ba55-4532366741b1 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:52.919643 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf010c9-08ab-412c-ba55-4532366741b1 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:52.919920 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf010c9-08ab-412c-ba55-4532366741b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:52.963391 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-fcf010c9-08ab-412c-ba55-4532366741b1 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:52.964653 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-fcf010c9-08ab-412c-ba55-4532366741b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:53.060260 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:53.096961 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260da95d-e59f-4474-bfbe-6996a4711d67 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:53.110441 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 362/704] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:53.133789 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf010c9-08ab-412c-ba55-4532366741b1 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:53.138252 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:53.140235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64451f-9dc6-4fec-b30d-d49db0329b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:53.141124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64451f-9dc6-4fec-b30d-d49db0329b83 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:53.141626 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64451f-9dc6-4fec-b30d-d49db0329b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:53.142117 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64451f-9dc6-4fec-b30d-d49db0329b83 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:53.143233 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c64451f-9dc6-4fec-b30d-d49db0329b83 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:53.143782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c64451f-9dc6-4fec-b30d-d49db0329b83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:53.146279 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c64451f-9dc6-4fec-b30d-d49db0329b83 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:53.175837 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 343/705] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Aug 3 04:54:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:53.182380 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64451f-9dc6-4fec-b30d-d49db0329b83 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:53.268631 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf010c9-08ab-412c-ba55-4532366741b1 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:53.330815 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ef6ba-08b3-45b4-84c1-3c39795a75f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:53.332228 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ef6ba-08b3-45b4-84c1-3c39795a75f8 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:53.332736 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ef6ba-08b3-45b4-84c1-3c39795a75f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:53.333659 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ef6ba-08b3-45b4-84c1-3c39795a75f8 None admin] PATH_INFO: `/v3/services/network` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:53.334797 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ef6ba-08b3-45b4-84c1-3c39795a75f8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:53.337139 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ef6ba-08b3-45b4-84c1-3c39795a75f8 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:53.357855 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-a41ef6ba-08b3-45b4-84c1-3c39795a75f8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 04:54:53.357855 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:53.357855 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 04:54:53.357855 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 04:54:53.357855 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 04:54:53.357855 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 04:54:53.357855 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 04:54:53.357855 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:54:53.357855 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 04:54:53.357855 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:53.357855 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:54:53.357855 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:53.357855 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:53.357855 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:53.357855 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:53.357855 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:53.357855 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:53.359029 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:53.359029 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:53.359029 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:53.359029 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:53.359029 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:53.359029 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:53.359029 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 04:54:53.359029 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 04:54:53.359029 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 04:54:53.359029 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 04:54:53.359029 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:53.359029 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:53.359029 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:53.359029 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:53.359029 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:53.359029 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:53.359940 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:53.359940 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:53.359940 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:53.359940 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:53.359940 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:53.359940 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:53.359940 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:53.359940 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:53.359940 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:53.359940 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:53.359940 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 04:54:53.359940 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:54:53.359940 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 04:54:53.359940 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:53.359940 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 363/706] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:54:53 2022] GET /identity/v3/services/network => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:54:53.371319 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ef6ba-08b3-45b4-84c1-3c39795a75f8 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:53.374952 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64451f-9dc6-4fec-b30d-d49db0329b83 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:53.412262 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:53.414285 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872466e-78fd-472b-b368-3ecd1c1b8daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:53.414935 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872466e-78fd-472b-b368-3ecd1c1b8daf None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:53.415221 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872466e-78fd-472b-b368-3ecd1c1b8daf None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:53.415493 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872466e-78fd-472b-b368-3ecd1c1b8daf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:53.489850 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:53.508381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ef6ba-08b3-45b4-84c1-3c39795a75f8 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:53.510189 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8872466e-78fd-472b-b368-3ecd1c1b8daf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:53.513100 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8872466e-78fd-472b-b368-3ecd1c1b8daf None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:53.553743 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e098a-b1c6-4517-8912-1b7b787051a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:53.554767 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e098a-b1c6-4517-8912-1b7b787051a3 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:53.555217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e098a-b1c6-4517-8912-1b7b787051a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:53.555670 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e098a-b1c6-4517-8912-1b7b787051a3 None admin] PATH_INFO: `/v3/services` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:53.556670 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051e098a-b1c6-4517-8912-1b7b787051a3 None admin] RBAC: Adding query filter params (name=network) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:53.557975 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051e098a-b1c6-4517-8912-1b7b787051a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:53.559560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051e098a-b1c6-4517-8912-1b7b787051a3 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:53.566088 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 364/707] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Aug 3 04:54:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:53.567604 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0089e-1014-4702-be11-532678ebc6aa None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:53.568101 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0089e-1014-4702-be11-532678ebc6aa None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:53.568540 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0089e-1014-4702-be11-532678ebc6aa None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:53.570289 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 365/708] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:53.571524 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0089e-1014-4702-be11-532678ebc6aa None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:53.658999 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 344/709] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:54:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 04:54:53.658999 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7a623-3688-4f50-bc31-6c6c43cdc228 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:53.658999 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7a623-3688-4f50-bc31-6c6c43cdc228 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:53.658999 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7a623-3688-4f50-bc31-6c6c43cdc228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:53.676948 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-1ed7a623-3688-4f50-bc31-6c6c43cdc228 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:53.678384 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1ed7a623-3688-4f50-bc31-6c6c43cdc228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:53.691485 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:53.702903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0089e-1014-4702-be11-532678ebc6aa None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:53.741740 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11b326-31ca-43f5-9911-ea990993e255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:53.742369 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11b326-31ca-43f5-9911-ea990993e255 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:53.742659 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11b326-31ca-43f5-9911-ea990993e255 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:53.742972 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11b326-31ca-43f5-9911-ea990993e255 None admin] PATH_INFO: `/v3/services` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:53.743652 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11b326-31ca-43f5-9911-ea990993e255 None admin] RBAC: Adding query filter params (type=network) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:53.743959 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11b326-31ca-43f5-9911-ea990993e255 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:53.745337 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11b326-31ca-43f5-9911-ea990993e255 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:53.751564 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 366/710] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Aug 3 04:54:53 2022] GET /identity/v3/services?type=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:53.753076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922123d9-66c0-46ad-80d6-8973966ea472 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:53.753586 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922123d9-66c0-46ad-80d6-8973966ea472 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:53.754161 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922123d9-66c0-46ad-80d6-8973966ea472 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:53.759864 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 367/711] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:53.760593 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922123d9-66c0-46ad-80d6-8973966ea472 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:53.771201 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 345/712] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:53.773121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d39f1-131f-423e-ab12-ab6cb1634f99 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:53.773901 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d39f1-131f-423e-ab12-ab6cb1634f99 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:53.774617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d39f1-131f-423e-ab12-ab6cb1634f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:53.796404 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-0d0d39f1-131f-423e-ab12-ab6cb1634f99 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:53.798249 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0d0d39f1-131f-423e-ab12-ab6cb1634f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:53.815475 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:53.897906 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922123d9-66c0-46ad-80d6-8973966ea472 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:53.904639 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 346/713] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:53.904639 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d39f1-131f-423e-ab12-ab6cb1634f99 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:53.962809 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3808370-75d0-4796-a59b-0ce281ee8dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:53.963416 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3808370-75d0-4796-a59b-0ce281ee8dfb None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:53.964629 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3808370-75d0-4796-a59b-0ce281ee8dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:53.964629 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3808370-75d0-4796-a59b-0ce281ee8dfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:53.977213 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3808370-75d0-4796-a59b-0ce281ee8dfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:53.978682 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3808370-75d0-4796-a59b-0ce281ee8dfb None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:54.000375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-c3808370-75d0-4796-a59b-0ce281ee8dfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:54.000375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:54.000375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:54.000375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:54.000375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:54.000375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:54.000375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:54.000375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:54.000375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:54.000375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:54.000375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:54.000375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:54.000375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:54.000375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:54.000375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:54.001378 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:54.001378 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:54.001378 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:54.001378 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:54.001378 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:54.001378 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:54.001378 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:54.001378 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:54.001378 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:54.001378 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:54.001378 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:54.001378 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:54.001378 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:54.001378 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:54.001378 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:54.001378 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:54.001378 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:54.002294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:54.002294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:54.002294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:54.002294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:54.003043 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 368/714] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:54.004112 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3808370-75d0-4796-a59b-0ce281ee8dfb None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:54.019395 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:54.042007 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d39f1-131f-423e-ab12-ab6cb1634f99 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:54.079626 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb2fd4b-dab6-4c17-a34f-4824e24022ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:54.080313 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2fd4b-dab6-4c17-a34f-4824e24022ff None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:54.080596 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2fd4b-dab6-4c17-a34f-4824e24022ff None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:54.080868 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2fd4b-dab6-4c17-a34f-4824e24022ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:54.091924 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb2fd4b-dab6-4c17-a34f-4824e24022ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:54.094177 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb2fd4b-dab6-4c17-a34f-4824e24022ff None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:54.102449 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:54.112387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-fdb2fd4b-dab6-4c17-a34f-4824e24022ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:54.112387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:54.112387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:54.112387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:54.112387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:54.112387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:54.112387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:54.112387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:54.112387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:54.112387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:54.112387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:54.112387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:54.112387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:54.112387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:54.112387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:54.113397 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:54.113397 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:54.113397 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:54.113397 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:54.113397 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:54.113397 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:54.113397 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:54.113397 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:54.113397 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:54.113397 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:54.113397 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:54.113397 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:54.113397 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:54.113397 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:54.113397 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:54.113397 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:54.113397 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:54.114466 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:54.114466 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:54.114466 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:54.114466 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:54.114466 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 347/715] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:54.115372 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb2fd4b-dab6-4c17-a34f-4824e24022ff None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:54.140401 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3808370-75d0-4796-a59b-0ce281ee8dfb None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:54.189412 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6272e0-0d02-43e1-8484-55478e1ee962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:54.190301 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6272e0-0d02-43e1-8484-55478e1ee962 None admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:54.190590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6272e0-0d02-43e1-8484-55478e1ee962 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:54.190891 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6272e0-0d02-43e1-8484-55478e1ee962 None admin] PATH_INFO: `/v3/services` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:54.192199 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6272e0-0d02-43e1-8484-55478e1ee962 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:54.193858 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6272e0-0d02-43e1-8484-55478e1ee962 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:54.229672 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb2fd4b-dab6-4c17-a34f-4824e24022ff None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:54.268426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a82a35-a2a5-4725-b937-6c365ed18237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:54.268829 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a82a35-a2a5-4725-b937-6c365ed18237 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:54.269114 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a82a35-a2a5-4725-b937-6c365ed18237 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:54.269440 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a82a35-a2a5-4725-b937-6c365ed18237 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:54.270228 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a82a35-a2a5-4725-b937-6c365ed18237 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:54.270534 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a82a35-a2a5-4725-b937-6c365ed18237 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:54.271987 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a82a35-a2a5-4725-b937-6c365ed18237 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:54.290994 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 348/717] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:54.290994 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a82a35-a2a5-4725-b937-6c365ed18237 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:54.301440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 369/717] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Aug 3 04:54:54 2022] POST /identity/v3/services => generated 251 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:54:54.301440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6272e0-0d02-43e1-8484-55478e1ee962 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:54.394874 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:54.449268 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:54.459607 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a82a35-a2a5-4725-b937-6c365ed18237 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:54.518923 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6272e0-0d02-43e1-8484-55478e1ee962 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:54.528662 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf14bd-52c7-42d3-a42c-457d547e8ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:54.528662 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf14bd-52c7-42d3-a42c-457d547e8ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:54.528662 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf14bd-52c7-42d3-a42c-457d547e8ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:54.528662 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf14bd-52c7-42d3-a42c-457d547e8ae8 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:54.529025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaf14bd-52c7-42d3-a42c-457d547e8ae8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:54.529025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaf14bd-52c7-42d3-a42c-457d547e8ae8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:54.531099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaf14bd-52c7-42d3-a42c-457d547e8ae8 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:54.548604 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 349/718] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:54.554743 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf14bd-52c7-42d3-a42c-457d547e8ae8 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:54.563474 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4028b6-4beb-4a1b-8c5f-11775e46e0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:54.564226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4028b6-4beb-4a1b-8c5f-11775e46e0af None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:54.564643 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4028b6-4beb-4a1b-8c5f-11775e46e0af None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:54.565038 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4028b6-4beb-4a1b-8c5f-11775e46e0af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:54.575749 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4028b6-4beb-4a1b-8c5f-11775e46e0af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:54.577796 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4028b6-4beb-4a1b-8c5f-11775e46e0af None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:54.591957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-4f4028b6-4beb-4a1b-8c5f-11775e46e0af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:54.591957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:54.591957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:54.591957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:54.591957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:54.591957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:54.591957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:54.591957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:54.591957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:54.591957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:54.591957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:54.591957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:54.591957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:54:54.591957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:54:54.591957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:54:54.593066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:54:54.593066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:54.593066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:54.593066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:54.593066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:54.593066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:54.593066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:54.593066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:54.593066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:54.593066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:54.593066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:54.593066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:54.593066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:54.593066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:54.593066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:54.593066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:54.593066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:54.593947 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:54:54.593947 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:54:54.593947 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:54:54.593947 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:54:54.593947 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:54:54.593947 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:54:54.593947 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:54.593947 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:54.593947 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 370/719] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:54:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:54.599212 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4028b6-4beb-4a1b-8c5f-11775e46e0af None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:54.692739 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf14bd-52c7-42d3-a42c-457d547e8ae8 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:54.715978 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:54.738044 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4028b6-4beb-4a1b-8c5f-11775e46e0af None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:54.749459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf157c06-bbe3-4c78-986d-cfc7176f4216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:54.750386 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf157c06-bbe3-4c78-986d-cfc7176f4216 None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:54.751011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf157c06-bbe3-4c78-986d-cfc7176f4216 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:54.751598 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf157c06-bbe3-4c78-986d-cfc7176f4216 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:54.752480 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf157c06-bbe3-4c78-986d-cfc7176f4216 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:54.754107 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf157c06-bbe3-4c78-986d-cfc7176f4216 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:54.790040 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:54.795475 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581eaed-abf4-4a56-b00d-2100111a8e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:54.796364 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581eaed-abf4-4a56-b00d-2100111a8e51 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:54.796649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581eaed-abf4-4a56-b00d-2100111a8e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:54.796935 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581eaed-abf4-4a56-b00d-2100111a8e51 None admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:54.797710 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f581eaed-abf4-4a56-b00d-2100111a8e51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:54.798108 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f581eaed-abf4-4a56-b00d-2100111a8e51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:54.799597 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f581eaed-abf4-4a56-b00d-2100111a8e51 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:54.849413 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 350/720] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 3 04:54:54 2022] POST /identity/v3/users => generated 254 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 04:54:54.944777 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 371/721] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:54:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:54:54.951346 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf157c06-bbe3-4c78-986d-cfc7176f4216 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:55.125337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf157c06-bbe3-4c78-986d-cfc7176f4216 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:55.161861 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f162ff-c784-425f-876d-47baf727d4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:55.162789 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f162ff-c784-425f-876d-47baf727d4c1 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:55.163254 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f162ff-c784-425f-876d-47baf727d4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:55.163744 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f162ff-c784-425f-876d-47baf727d4c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:55.164692 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f162ff-c784-425f-876d-47baf727d4c1 None admin] RBAC: Adding query filter params (role.id=e39d4ecea4c64d5794e1079c9be3ad36, scope.project.id=cf3763a37383456184a9ca9351b4ce03, user.id=a3fa4a6c16274c18849869b55b33ac62) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:55.165167 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f162ff-c784-425f-876d-47baf727d4c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:55.166763 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f162ff-c784-425f-876d-47baf727d4c1 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:55.179004 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 351/722] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:54:54 2022] GET /identity/v3/role_assignments?user.id=a3fa4a6c16274c18849869b55b33ac62&scope.project.id=cf3763a37383456184a9ca9351b4ce03&role.id=e39d4ecea4c64d5794e1079c9be3ad36 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:55.345048 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db249b47-3081-43ef-9027-277525de0b49 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:55.345330 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db249b47-3081-43ef-9027-277525de0b49 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:55.345609 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db249b47-3081-43ef-9027-277525de0b49 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:55.347136 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 372/723] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:55.352359 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceefa23-b6d4-41e7-b9be-cea7b3c1ec94 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:55.352878 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceefa23-b6d4-41e7-b9be-cea7b3c1ec94 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:55.353162 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceefa23-b6d4-41e7-b9be-cea7b3c1ec94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:55.376417 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-aceefa23-b6d4-41e7-b9be-cea7b3c1ec94 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:55.377967 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-aceefa23-b6d4-41e7-b9be-cea7b3c1ec94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:55.412378 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:55.581796 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 352/724] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:55 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 04:54:55.617036 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db249b47-3081-43ef-9027-277525de0b49 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:55.750351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db249b47-3081-43ef-9027-277525de0b49 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:55.786030 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394bf39c-2016-4caf-97a6-07bb3934d843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:55.786751 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394bf39c-2016-4caf-97a6-07bb3934d843 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:55.787062 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394bf39c-2016-4caf-97a6-07bb3934d843 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:55.787392 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394bf39c-2016-4caf-97a6-07bb3934d843 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:55.798298 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394bf39c-2016-4caf-97a6-07bb3934d843 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:55.803874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-394bf39c-2016-4caf-97a6-07bb3934d843 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:54:55.803874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:55.803874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:55.803874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:55.803874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:55.803874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:55.803874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:55.803874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:55.803874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:55.803874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:55.803874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:55.803874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:55.803874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:55.803874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:55.803874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:55.803874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:55.804913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:55.804913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:55.804913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:55.804913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:55.804913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:55.804913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:55.804913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:55.804913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:55.804913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:55.804913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:55.804913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:55.804913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:55.804913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:55.804913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:55.804913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:55.804913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:55.806164 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:55.806164 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:55.806164 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:55.806164 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:55.806164 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:55.806164 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:55.806164 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:55.806164 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:55.806164 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:55.806164 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:55.806164 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:54:55.806164 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:55.806164 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 373/725] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:54:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:55.811260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceefa23-b6d4-41e7-b9be-cea7b3c1ec94 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:56.006721 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceefa23-b6d4-41e7-b9be-cea7b3c1ec94 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:56.104847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b21ce-090b-4fa9-ac0d-b097929295b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:56.105486 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b21ce-090b-4fa9-ac0d-b097929295b3 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:56.105770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b21ce-090b-4fa9-ac0d-b097929295b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:56.106024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b21ce-090b-4fa9-ac0d-b097929295b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:56.106762 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b21ce-090b-4fa9-ac0d-b097929295b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:56.107091 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b21ce-090b-4fa9-ac0d-b097929295b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:56.111967 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b21ce-090b-4fa9-ac0d-b097929295b3 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:56.134634 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 353/726] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:54:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:56.144668 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394bf39c-2016-4caf-97a6-07bb3934d843 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:56.222391 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:56.298710 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394bf39c-2016-4caf-97a6-07bb3934d843 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:56.303170 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194dec1d-9af7-4bde-84a9-11af8d71c67c None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:56.306227 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194dec1d-9af7-4bde-84a9-11af8d71c67c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:56.306227 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194dec1d-9af7-4bde-84a9-11af8d71c67c None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:56.306227 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 354/727] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:56.312658 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fd0d4-353f-457c-95f1-212d71119264 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:56.313012 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fd0d4-353f-457c-95f1-212d71119264 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:56.313491 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fd0d4-353f-457c-95f1-212d71119264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:56.349148 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-2a1fd0d4-353f-457c-95f1-212d71119264 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:56.350366 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2a1fd0d4-353f-457c-95f1-212d71119264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:56.374097 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da79d3-8abe-4d8d-83e4-30be5890742a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:56.374756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da79d3-8abe-4d8d-83e4-30be5890742a None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:56.375073 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da79d3-8abe-4d8d-83e4-30be5890742a None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:56.375357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da79d3-8abe-4d8d-83e4-30be5890742a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:56.402989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da79d3-8abe-4d8d-83e4-30be5890742a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:56.404312 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:56.409643 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da79d3-8abe-4d8d-83e4-30be5890742a None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:56.447117 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-b9da79d3-8abe-4d8d-83e4-30be5890742a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:56.447117 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:56.447117 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:56.447117 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:56.447117 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:56.447117 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:56.447117 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:56.447117 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:56.447117 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:56.447117 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:56.447117 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:56.447117 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:56.447117 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:56.447117 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:56.447117 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:56.448079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:56.448079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:56.448079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:56.448079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:56.448079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:56.448079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:56.448079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:56.448079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:56.448079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:56.448079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:56.448079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:56.448079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:56.448079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:56.448079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:56.448079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:56.448079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:56.448079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:56.448958 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:56.448958 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:56.448958 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:56.448958 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:56.451168 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 374/728] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:56.457250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da79d3-8abe-4d8d-83e4-30be5890742a None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:56.515492 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 355/729] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:56 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 04:54:56.539195 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1fd0d4-353f-457c-95f1-212d71119264 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:56.650742 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1fd0d4-353f-457c-95f1-212d71119264 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:56.679161 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da79d3-8abe-4d8d-83e4-30be5890742a None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:56.683181 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:56.695733 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3b9a1-e01c-4fa1-9172-354097a76cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:56.696359 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3b9a1-e01c-4fa1-9172-354097a76cdc None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:56.696633 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3b9a1-e01c-4fa1-9172-354097a76cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:56.696908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3b9a1-e01c-4fa1-9172-354097a76cdc None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:56.697620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a3b9a1-e01c-4fa1-9172-354097a76cdc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:56.699007 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a3b9a1-e01c-4fa1-9172-354097a76cdc None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:56.738088 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c923b8-c88c-488b-ac36-3e00eb75ac13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:56.738812 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c923b8-c88c-488b-ac36-3e00eb75ac13 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:56.739122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c923b8-c88c-488b-ac36-3e00eb75ac13 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:56.739443 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c923b8-c88c-488b-ac36-3e00eb75ac13 None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:56.740332 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c923b8-c88c-488b-ac36-3e00eb75ac13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:56.740636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c923b8-c88c-488b-ac36-3e00eb75ac13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:56.742088 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c923b8-c88c-488b-ac36-3e00eb75ac13 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:56.756198 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 375/730] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:56.763001 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c923b8-c88c-488b-ac36-3e00eb75ac13 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:56.775392 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 356/731] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Aug 3 04:54:56 2022] POST /identity/v3/services => generated 240 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 04:54:56.890612 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446c66b-619e-45fc-bb3b-f91af59f4ed4 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:56.890896 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c923b8-c88c-488b-ac36-3e00eb75ac13 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:56.890986 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446c66b-619e-45fc-bb3b-f91af59f4ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:56.891217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446c66b-619e-45fc-bb3b-f91af59f4ed4 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:56.892912 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 357/732] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:54:56.898871 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f339b83-11dd-4672-b6ec-284d003061ad None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:56.899177 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f339b83-11dd-4672-b6ec-284d003061ad None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:56.899439 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f339b83-11dd-4672-b6ec-284d003061ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:56.941782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-4f339b83-11dd-4672-b6ec-284d003061ad None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:56.943152 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-4f339b83-11dd-4672-b6ec-284d003061ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:56.964130 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be3169-91f4-4633-b906-95d696f6e33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:56.964786 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be3169-91f4-4633-b906-95d696f6e33d None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:56.965063 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be3169-91f4-4633-b906-95d696f6e33d None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:56.965330 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be3169-91f4-4633-b906-95d696f6e33d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:56.981279 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:56.987278 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be3169-91f4-4633-b906-95d696f6e33d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:56.992187 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be3169-91f4-4633-b906-95d696f6e33d None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:57.015542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-84be3169-91f4-4633-b906-95d696f6e33d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 03 04:54:57.015542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:57.015542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:57.015542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:57.015542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:57.015542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:57.015542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:57.015542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:57.015542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:57.015542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:57.015542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:57.015542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:57.015542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:54:57.015542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:54:57.015542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:54:57.015542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:54:57.021020 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:57.021020 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:57.021020 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:54:57.021020 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:57.021020 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:54:57.021020 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:57.021020 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:57.021020 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:57.021020 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:57.021020 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:57.021020 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:57.021020 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:57.021020 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:57.021020 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:57.021020 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:57.021020 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:57.021020 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:57.021020 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:57.022397 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:57.022397 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:57.022397 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:54:57.022397 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:54:57.022397 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:54:57.022397 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:54:57.022397 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:54:57.022397 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:54:57.022397 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 03 04:54:57.022397 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:57.022397 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 376/733] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:54:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:57.026879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be3169-91f4-4633-b906-95d696f6e33d None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:57.051204 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:57.068332 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 358/734] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 04:54:57.096094 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f339b83-11dd-4672-b6ec-284d003061ad None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:57.163534 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be3169-91f4-4633-b906-95d696f6e33d None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:57.210717 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d1990-08ba-49b0-a132-ce44c7324c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:57.211294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d1990-08ba-49b0-a132-ce44c7324c3f None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:57.211626 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d1990-08ba-49b0-a132-ce44c7324c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:57.211908 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d1990-08ba-49b0-a132-ce44c7324c3f None admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:57.212708 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d1990-08ba-49b0-a132-ce44c7324c3f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:57.213017 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d1990-08ba-49b0-a132-ce44c7324c3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:57.220660 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d1990-08ba-49b0-a132-ce44c7324c3f None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:57.221977 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f339b83-11dd-4672-b6ec-284d003061ad None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:57.253199 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 377/735] 10.222.0.40 () {56 vars in 1135 bytes} [Wed Aug 3 04:54:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:57.258803 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d1990-08ba-49b0-a132-ce44c7324c3f None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:57.267521 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f693ff7-1164-4ac8-98e3-c07a58f00027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:57.268174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f693ff7-1164-4ac8-98e3-c07a58f00027 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:57.268452 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f693ff7-1164-4ac8-98e3-c07a58f00027 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:57.268725 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f693ff7-1164-4ac8-98e3-c07a58f00027 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:57.269361 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f693ff7-1164-4ac8-98e3-c07a58f00027 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:57.270828 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f693ff7-1164-4ac8-98e3-c07a58f00027 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:57.290134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-8f693ff7-1164-4ac8-98e3-c07a58f00027 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 04:54:57.290134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:57.290134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 04:54:57.290134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 04:54:57.290134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 04:54:57.290134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 04:54:57.290134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 04:54:57.290134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:54:57.290134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 04:54:57.290134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:57.290134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:54:57.290134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:57.290134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:57.290134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:57.290134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:57.290134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:57.292438 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:57.292438 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:57.292438 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:57.292438 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:57.292438 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:57.292438 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:57.292438 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:57.292438 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 04:54:57.292438 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 04:54:57.292438 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 04:54:57.292438 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 04:54:57.292438 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:57.292438 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:57.292438 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:57.292438 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:57.292438 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:57.294408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:57.294408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:57.294408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:57.294408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:57.294408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:57.294408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:57.294408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:57.294408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:57.294408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:57.294408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:57.294408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:57.294408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 04:54:57.294408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:54:57.294408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 04:54:57.294408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:57.299985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 359/736] 10.222.0.40 () {56 vars in 1129 bytes} [Wed Aug 3 04:54:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:54:57.302866 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f693ff7-1164-4ac8-98e3-c07a58f00027 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:57.336426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:57.360909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:57.422582 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d1990-08ba-49b0-a132-ce44c7324c3f None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:57.483585 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192cef1a-d64d-4458-9bb9-dfca29b9bdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:57.484193 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192cef1a-d64d-4458-9bb9-dfca29b9bdfc None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:57.484474 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192cef1a-d64d-4458-9bb9-dfca29b9bdfc None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:57.484758 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192cef1a-d64d-4458-9bb9-dfca29b9bdfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:57.517658 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f693ff7-1164-4ac8-98e3-c07a58f00027 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:57.566093 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827696b7-d15e-4a14-98e1-be36c3d0ffb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:57.566765 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827696b7-d15e-4a14-98e1-be36c3d0ffb7 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:57.567087 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827696b7-d15e-4a14-98e1-be36c3d0ffb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:57.567352 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827696b7-d15e-4a14-98e1-be36c3d0ffb7 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:57.568042 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827696b7-d15e-4a14-98e1-be36c3d0ffb7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:57.568352 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827696b7-d15e-4a14-98e1-be36c3d0ffb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:57.569738 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827696b7-d15e-4a14-98e1-be36c3d0ffb7 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:57.575219 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 360/737] 10.222.0.40 () {56 vars in 1124 bytes} [Wed Aug 3 04:54:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:57.580856 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827696b7-d15e-4a14-98e1-be36c3d0ffb7 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:57.622829 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192cef1a-d64d-4458-9bb9-dfca29b9bdfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:57.624467 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192cef1a-d64d-4458-9bb9-dfca29b9bdfc None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:57.670969 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:57.687323 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827696b7-d15e-4a14-98e1-be36c3d0ffb7 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:57.700158 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:57.723156 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97435aae-09dc-479e-8aa2-96fa0ab1016d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:57.723859 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97435aae-09dc-479e-8aa2-96fa0ab1016d None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:57.724155 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97435aae-09dc-479e-8aa2-96fa0ab1016d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:57.724508 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97435aae-09dc-479e-8aa2-96fa0ab1016d None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:57.725208 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97435aae-09dc-479e-8aa2-96fa0ab1016d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:57.725521 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97435aae-09dc-479e-8aa2-96fa0ab1016d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:57.727064 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97435aae-09dc-479e-8aa2-96fa0ab1016d None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:57.735415 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 361/738] 10.222.0.40 () {56 vars in 1124 bytes} [Wed Aug 3 04:54:57 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:57.737870 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 378/739] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:54:57 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 03 04:54:57.743052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97435aae-09dc-479e-8aa2-96fa0ab1016d None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:57.744966 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192cef1a-d64d-4458-9bb9-dfca29b9bdfc None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:57.849745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97435aae-09dc-479e-8aa2-96fa0ab1016d None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:57.849950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192cef1a-d64d-4458-9bb9-dfca29b9bdfc None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:57.888057 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16bffc7-9701-444a-b59d-2ae32fda48c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:57.888308 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dad7c1-490a-44b7-9cfe-6f1ccc92f9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:57.888702 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16bffc7-9701-444a-b59d-2ae32fda48c1 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:57.889019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16bffc7-9701-444a-b59d-2ae32fda48c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:57.889301 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16bffc7-9701-444a-b59d-2ae32fda48c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:57.889857 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dad7c1-490a-44b7-9cfe-6f1ccc92f9f0 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:57.890351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dad7c1-490a-44b7-9cfe-6f1ccc92f9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:57.890823 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dad7c1-490a-44b7-9cfe-6f1ccc92f9f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:57.891818 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dad7c1-490a-44b7-9cfe-6f1ccc92f9f0 None admin] RBAC: Adding query filter params (interface=public, service_id=6193268ac2194437881f6ba35624082c, region_id=RegionOne) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:57.892323 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dad7c1-490a-44b7-9cfe-6f1ccc92f9f0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:57.893884 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dad7c1-490a-44b7-9cfe-6f1ccc92f9f0 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:57.900648 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16bffc7-9701-444a-b59d-2ae32fda48c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:57.902089 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16bffc7-9701-444a-b59d-2ae32fda48c1 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:57.902747 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 379/740] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Aug 3 04:54:57 2022] GET /identity/v3/endpoints?service_id=6193268ac2194437881f6ba35624082c&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:57.904070 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80295840-fb85-4dc1-bc6c-3e84ffadab61 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:57.904365 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80295840-fb85-4dc1-bc6c-3e84ffadab61 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:57.905178 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80295840-fb85-4dc1-bc6c-3e84ffadab61 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:57.906537 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 380/741] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:57.907553 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce09d266-4314-4660-9a50-7798330ae60f None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:57.907846 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce09d266-4314-4660-9a50-7798330ae60f None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:57.908123 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce09d266-4314-4660-9a50-7798330ae60f None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:57.909424 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 381/742] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:57.912093 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:57.914821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93fc307-aa75-4d63-9d56-69a4dc10585d None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:57.915169 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93fc307-aa75-4d63-9d56-69a4dc10585d None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:57.915449 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93fc307-aa75-4d63-9d56-69a4dc10585d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:57.926813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-e16bffc7-9701-444a-b59d-2ae32fda48c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:57.926813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:57.926813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:57.926813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:57.926813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:57.926813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:57.926813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:57.926813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:57.926813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:57.926813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:57.926813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:57.926813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:57.926813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:57.926813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:57.926813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:57.928202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:57.928202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:57.928202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:57.928202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:57.928202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:57.928202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:57.928202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:57.928202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:57.928202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:57.928202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:57.928202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:57.928202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:57.928202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:57.928202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:57.928202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:57.928202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:57.928202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:57.929087 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:57.929087 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:57.929087 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:57.929087 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:57.932711 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 362/743] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:57.933922 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16bffc7-9701-444a-b59d-2ae32fda48c1 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:57.936043 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-a93fc307-aa75-4d63-9d56-69a4dc10585d None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:57.937278 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-a93fc307-aa75-4d63-9d56-69a4dc10585d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:57.966281 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:58.015170 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 382/744] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:54:58.016953 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa64d7e-a52c-49fc-933a-ff74bf239d98 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:58.017247 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa64d7e-a52c-49fc-933a-ff74bf239d98 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:58.017532 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa64d7e-a52c-49fc-933a-ff74bf239d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:58.040747 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-7aa64d7e-a52c-49fc-933a-ff74bf239d98 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:58.041931 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-7aa64d7e-a52c-49fc-933a-ff74bf239d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:58.043136 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16bffc7-9701-444a-b59d-2ae32fda48c1 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:58.087992 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c57bc2-5b80-4a32-ac25-72a91af13d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:58.088617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c57bc2-5b80-4a32-ac25-72a91af13d4a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:58.088901 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c57bc2-5b80-4a32-ac25-72a91af13d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:58.089180 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c57bc2-5b80-4a32-ac25-72a91af13d4a None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:58.089833 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c57bc2-5b80-4a32-ac25-72a91af13d4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:58.091231 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c57bc2-5b80-4a32-ac25-72a91af13d4a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:58.097617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 363/745] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Aug 3 04:54:57 2022] GET /identity/v3/services => generated 772 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:58.110073 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c57bc2-5b80-4a32-ac25-72a91af13d4a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:58.132405 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 383/746] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:54:58.132405 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa64d7e-a52c-49fc-933a-ff74bf239d98 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:58.216124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:58.261342 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:58.268733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa64d7e-a52c-49fc-933a-ff74bf239d98 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:58.275347 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c57bc2-5b80-4a32-ac25-72a91af13d4a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:58.313961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739467f9-fcf3-43c8-a47c-368a27c53725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:58.314631 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739467f9-fcf3-43c8-a47c-368a27c53725 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:58.314946 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739467f9-fcf3-43c8-a47c-368a27c53725 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:58.315286 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739467f9-fcf3-43c8-a47c-368a27c53725 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:58.328685 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b8aa4-94be-4231-9d2c-e5ec8f341370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:58.330023 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b8aa4-94be-4231-9d2c-e5ec8f341370 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:58.330208 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739467f9-fcf3-43c8-a47c-368a27c53725 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:58.330688 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b8aa4-94be-4231-9d2c-e5ec8f341370 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:58.331220 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b8aa4-94be-4231-9d2c-e5ec8f341370 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:58.332144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b8aa4-94be-4231-9d2c-e5ec8f341370 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:58.332656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b8aa4-94be-4231-9d2c-e5ec8f341370 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:58.334475 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b8aa4-94be-4231-9d2c-e5ec8f341370 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:58.335502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-739467f9-fcf3-43c8-a47c-368a27c53725 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:54:58.335502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:58.335502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:58.335502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:58.335502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:58.335502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:58.335502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:58.335502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:58.335502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:58.335502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:58.335502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:58.335502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:58.335502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:58.335502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:58.335502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:58.335502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:58.336430 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:58.336430 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:58.336430 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:58.336430 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:58.336430 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:58.336430 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:58.336430 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:58.336430 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:58.336430 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:58.336430 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:58.336430 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:58.336430 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:58.336430 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:58.336430 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:58.336430 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:58.336430 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:58.337290 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:58.337290 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:58.337290 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:58.337290 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:58.337290 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:58.337290 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:58.337290 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:58.337290 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:58.337290 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:58.337290 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:58.337290 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:54:58.337290 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:58.337290 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 384/747] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:54:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:58.342726 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739467f9-fcf3-43c8-a47c-368a27c53725 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:58.352750 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 364/748] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:58.359209 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b8aa4-94be-4231-9d2c-e5ec8f341370 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:58.462211 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739467f9-fcf3-43c8-a47c-368a27c53725 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:58.497352 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b8aa4-94be-4231-9d2c-e5ec8f341370 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:58.500900 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c308ca22-ab15-4272-8307-1d33f2f88c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:58.501816 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308ca22-ab15-4272-8307-1d33f2f88c69 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:58.502304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308ca22-ab15-4272-8307-1d33f2f88c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:58.502784 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308ca22-ab15-4272-8307-1d33f2f88c69 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:58.515176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c308ca22-ab15-4272-8307-1d33f2f88c69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:54:58.518888 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:58.521600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-c308ca22-ab15-4272-8307-1d33f2f88c69 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:58.521600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:58.521600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:58.521600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:58.521600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:58.521600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:58.521600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:58.521600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:58.521600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:58.521600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:58.521600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:58.521600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:58.521600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:54:58.521600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:54:58.521600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:54:58.521600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:54:58.522559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:54:58.522559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:54:58.522559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:54:58.522559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:54:58.522559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:58.522559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:58.522559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:58.522559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:58.522559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:58.522559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:58.522559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:58.522559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:58.522559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:58.522559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:58.522559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:58.522559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:58.524086 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:58.524086 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:58.524086 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:58.524086 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:58.524086 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:54:58.524086 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:54:58.524086 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:54:58.524086 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:54:58.524086 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:54:58.524086 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:54:58.524086 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:54:58.524086 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:58.527376 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 385/749] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 3 04:54:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:58.528544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c308ca22-ab15-4272-8307-1d33f2f88c69 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:58.537837 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f56554-07fc-4e4c-9fa3-4d3a9abb27df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:58.539869 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f56554-07fc-4e4c-9fa3-4d3a9abb27df None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:58.539869 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f56554-07fc-4e4c-9fa3-4d3a9abb27df None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:58.539869 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f56554-07fc-4e4c-9fa3-4d3a9abb27df None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:58.542657 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f56554-07fc-4e4c-9fa3-4d3a9abb27df None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:58.543003 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f56554-07fc-4e4c-9fa3-4d3a9abb27df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:58.544831 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f56554-07fc-4e4c-9fa3-4d3a9abb27df None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:58.548547 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:58.561836 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 365/750] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:54:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:58.561836 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f56554-07fc-4e4c-9fa3-4d3a9abb27df None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:58.663220 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c308ca22-ab15-4272-8307-1d33f2f88c69 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:58.693157 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f56554-07fc-4e4c-9fa3-4d3a9abb27df None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:58.704274 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9baf07e-e476-4fda-9f72-1165ee920109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:58.704925 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9baf07e-e476-4fda-9f72-1165ee920109 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:58.705211 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9baf07e-e476-4fda-9f72-1165ee920109 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:58.705492 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9baf07e-e476-4fda-9f72-1165ee920109 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:58.712983 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9baf07e-e476-4fda-9f72-1165ee920109 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:58.714371 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9baf07e-e476-4fda-9f72-1165ee920109 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:58.726796 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-d9baf07e-e476-4fda-9f72-1165ee920109 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:58.726796 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:58.726796 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:58.726796 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:58.726796 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:58.726796 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:58.726796 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:58.726796 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:58.726796 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:58.726796 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:58.726796 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:58.726796 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:58.726796 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:54:58.726796 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:54:58.726796 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:54:58.727801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:54:58.727801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:58.727801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:58.727801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:58.727801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:58.727801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:58.727801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:58.727801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:58.727801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:58.727801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:58.727801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:58.727801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:58.727801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:58.727801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:58.727801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:58.727801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:58.727801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:58.728725 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:54:58.728725 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:54:58.728725 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:54:58.728725 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:54:58.728725 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:54:58.728725 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:54:58.728725 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:54:58.728725 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:58.733593 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 386/751] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:54:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:58.734449 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9baf07e-e476-4fda-9f72-1165ee920109 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:58.744143 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bab726-2978-4860-a082-f875c54eb917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:58.747375 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bab726-2978-4860-a082-f875c54eb917 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:58.747910 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bab726-2978-4860-a082-f875c54eb917 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:58.751286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bab726-2978-4860-a082-f875c54eb917 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:58.752321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bab726-2978-4860-a082-f875c54eb917 None admin] RBAC: Adding query filter params (name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:58.752845 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bab726-2978-4860-a082-f875c54eb917 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:58.754456 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bab726-2978-4860-a082-f875c54eb917 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:58.768407 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 366/752] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 3 04:54:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:58.769708 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bab726-2978-4860-a082-f875c54eb917 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:58.817354 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:58.822649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:58.849013 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9baf07e-e476-4fda-9f72-1165ee920109 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:58.880089 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bab726-2978-4860-a082-f875c54eb917 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:58.887133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d3ef4-ecfe-4e4b-af7e-ba1fcd6c3bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:58.887781 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d3ef4-ecfe-4e4b-af7e-ba1fcd6c3bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:58.888069 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d3ef4-ecfe-4e4b-af7e-ba1fcd6c3bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:58.888349 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d3ef4-ecfe-4e4b-af7e-ba1fcd6c3bf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:58.901139 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d3ef4-ecfe-4e4b-af7e-ba1fcd6c3bf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:58.902615 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d3ef4-ecfe-4e4b-af7e-ba1fcd6c3bf0 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:58.918354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-f84d3ef4-ecfe-4e4b-af7e-ba1fcd6c3bf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:58.918354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:58.918354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:58.918354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:58.918354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:58.918354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:58.918354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:58.918354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:58.918354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:58.918354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:58.918354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:58.918354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:58.918354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:58.918354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:58.918354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:58.919365 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:58.919365 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:58.919365 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:58.919365 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:58.919365 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:58.919365 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:58.919365 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:58.919365 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:58.919365 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:58.919365 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:58.919365 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:58.919365 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:58.919365 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:58.919365 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:58.919365 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:58.919365 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:58.919365 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:58.921435 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:58.921435 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:58.921435 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:58.921435 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:58.921435 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 387/753] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:58.921435 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d3ef4-ecfe-4e4b-af7e-ba1fcd6c3bf0 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:58.923342 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437192c-cd80-4451-b26d-fdd75c08abaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:58.924232 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437192c-cd80-4451-b26d-fdd75c08abaf None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:58.924867 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437192c-cd80-4451-b26d-fdd75c08abaf None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:58.925352 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437192c-cd80-4451-b26d-fdd75c08abaf None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:58.926320 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5437192c-cd80-4451-b26d-fdd75c08abaf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:58.926857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5437192c-cd80-4451-b26d-fdd75c08abaf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:58.928542 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5437192c-cd80-4451-b26d-fdd75c08abaf None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:58.942820 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 367/754] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:54:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:58.944097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437192c-cd80-4451-b26d-fdd75c08abaf None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:59.037101 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d3ef4-ecfe-4e4b-af7e-ba1fcd6c3bf0 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:59.050328 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437192c-cd80-4451-b26d-fdd75c08abaf None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:59.052182 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:59.064348 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:59.073571 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac75cfa-4d88-48dc-a2d2-bf252582df82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:59.074306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac75cfa-4d88-48dc-a2d2-bf252582df82 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:59.074507 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac75cfa-4d88-48dc-a2d2-bf252582df82 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:59.074769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac75cfa-4d88-48dc-a2d2-bf252582df82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:59.086778 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac75cfa-4d88-48dc-a2d2-bf252582df82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:59.088430 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac75cfa-4d88-48dc-a2d2-bf252582df82 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:59.088913 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d75f3-ec92-4809-a398-40550a65c7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:59.089811 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d75f3-ec92-4809-a398-40550a65c7eb None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:59.090290 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d75f3-ec92-4809-a398-40550a65c7eb None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:59.090752 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d75f3-ec92-4809-a398-40550a65c7eb None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:59.091655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44d75f3-ec92-4809-a398-40550a65c7eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:59.093646 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44d75f3-ec92-4809-a398-40550a65c7eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:59.095320 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44d75f3-ec92-4809-a398-40550a65c7eb None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:59.104948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-5ac75cfa-4d88-48dc-a2d2-bf252582df82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:59.104948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:59.104948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:59.104948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:59.104948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:59.104948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:59.104948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:59.104948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:59.104948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:59.104948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:59.104948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:59.104948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:59.104948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:54:59.104948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:54:59.104948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:54:59.105897 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:54:59.105897 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:59.105897 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:59.105897 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:59.105897 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:59.105897 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:59.105897 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:59.105897 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:59.105897 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:59.105897 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:59.105897 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:59.105897 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:59.105897 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:59.105897 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:59.105897 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:59.105897 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:59.105897 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:59.106804 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:54:59.106804 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:54:59.106804 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:54:59.106804 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:59.111424 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 368/756] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:59.111549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 388/756] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:54:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:59.112551 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac75cfa-4d88-48dc-a2d2-bf252582df82 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:59.112891 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d75f3-ec92-4809-a398-40550a65c7eb None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:59.227904 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d75f3-ec92-4809-a398-40550a65c7eb None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:59.233301 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac75cfa-4d88-48dc-a2d2-bf252582df82 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:59.264481 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d81f1-ba42-4e77-9328-6d5202abfd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:59.265102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d81f1-ba42-4e77-9328-6d5202abfd75 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:59.265388 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d81f1-ba42-4e77-9328-6d5202abfd75 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:59.265669 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d81f1-ba42-4e77-9328-6d5202abfd75 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:59.266368 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d81f1-ba42-4e77-9328-6d5202abfd75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:59.266681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d81f1-ba42-4e77-9328-6d5202abfd75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:59.268083 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d81f1-ba42-4e77-9328-6d5202abfd75 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:59.269303 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf018f-fb5c-4b27-aac7-81f7496568b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:59.269923 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf018f-fb5c-4b27-aac7-81f7496568b7 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:59.270209 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf018f-fb5c-4b27-aac7-81f7496568b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:59.270781 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf018f-fb5c-4b27-aac7-81f7496568b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:59.271349 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf018f-fb5c-4b27-aac7-81f7496568b7 None admin] RBAC: Adding query filter params (role.id=e39d4ecea4c64d5794e1079c9be3ad36, scope.project.id=cf3763a37383456184a9ca9351b4ce03, user.id=e1014bcb4d36441aa46042941c9a63d0) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:59.271662 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf018f-fb5c-4b27-aac7-81f7496568b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:59.273358 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf018f-fb5c-4b27-aac7-81f7496568b7 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:59.285870 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 369/757] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:59.285870 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d81f1-ba42-4e77-9328-6d5202abfd75 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:59.286180 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 389/758] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:54:59 2022] GET /identity/v3/role_assignments?user.id=e1014bcb4d36441aa46042941c9a63d0&scope.project.id=cf3763a37383456184a9ca9351b4ce03&role.id=e39d4ecea4c64d5794e1079c9be3ad36 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:59.291004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef68a7-0cb1-47a8-b678-e92bc7a12919 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:59.291004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef68a7-0cb1-47a8-b678-e92bc7a12919 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:59.291004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef68a7-0cb1-47a8-b678-e92bc7a12919 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:59.291004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 390/759] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:54:59.291004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef68a7-0cb1-47a8-b678-e92bc7a12919 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:59.328615 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:59.367225 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:59.424639 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef68a7-0cb1-47a8-b678-e92bc7a12919 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:59.435860 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d81f1-ba42-4e77-9328-6d5202abfd75 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:59.470226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94b7a0-825f-43c7-9fe9-58d472fea830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:59.470974 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94b7a0-825f-43c7-9fe9-58d472fea830 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:59.471265 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94b7a0-825f-43c7-9fe9-58d472fea830 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:59.471543 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94b7a0-825f-43c7-9fe9-58d472fea830 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:59.480140 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e94b7a0-825f-43c7-9fe9-58d472fea830 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:59.481643 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e94b7a0-825f-43c7-9fe9-58d472fea830 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:59.486096 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a228e-0c43-49ce-acfd-8ee2587ea255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:59.487041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a228e-0c43-49ce-acfd-8ee2587ea255 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:59.487507 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a228e-0c43-49ce-acfd-8ee2587ea255 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:59.488209 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a228e-0c43-49ce-acfd-8ee2587ea255 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:59.493199 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-7e94b7a0-825f-43c7-9fe9-58d472fea830 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 04:54:59.493199 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:59.493199 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:59.493199 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:59.493199 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:59.493199 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:59.493199 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:59.493199 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:59.493199 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:59.493199 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:59.493199 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:59.493199 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:59.493199 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:54:59.493199 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:54:59.493199 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:54:59.493199 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:54:59.494141 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:59.494141 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:59.494141 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:54:59.494141 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:59.494141 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:54:59.494141 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:59.494141 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:59.494141 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:59.494141 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:59.494141 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:59.494141 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:59.494141 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:59.494141 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:59.494141 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:59.494141 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:59.494141 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:59.494141 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:59.494141 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:59.495314 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:59.495314 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:59.495314 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:54:59.495314 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:54:59.495314 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:54:59.495314 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:54:59.495314 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:54:59.495314 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:54:59.495314 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 04:54:59.495314 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:54:59.499155 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 391/760] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Aug 3 04:54:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:59.499469 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a228e-0c43-49ce-acfd-8ee2587ea255 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:59.500424 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c506f6b-58e8-45e7-a4b9-57f4565bc243 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:59.500715 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c506f6b-58e8-45e7-a4b9-57f4565bc243 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:59.500995 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c506f6b-58e8-45e7-a4b9-57f4565bc243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:59.501796 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a228e-0c43-49ce-acfd-8ee2587ea255 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:59.517174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-2f8a228e-0c43-49ce-acfd-8ee2587ea255 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 04:54:59.517174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:54:59.517174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:54:59.517174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:54:59.517174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:54:59.517174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:54:59.517174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:54:59.517174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:54:59.517174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:54:59.517174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:54:59.517174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:54:59.517174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:54:59.517174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:54:59.517174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:54:59.517174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:54:59.517174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:54:59.518187 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:54:59.518187 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:54:59.518187 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:54:59.518187 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:59.518187 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:54:59.518187 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:54:59.518187 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:54:59.518187 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:54:59.518187 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:54:59.518187 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:54:59.518187 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:54:59.518187 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:54:59.518187 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:54:59.518187 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:54:59.518187 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:54:59.518187 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:54:59.518187 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:54:59.518187 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:54:59.519608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:54:59.519608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:54:59.519608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:54:59.519608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:54:59.519608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:54:59.519608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:54:59.519608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:54:59.519608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:54:59.519608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 04:54:59.519608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:54:59.521689 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-4c506f6b-58e8-45e7-a4b9-57f4565bc243 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:54:59.522093 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 370/761] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Aug 3 04:54:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:54:59.523375 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a228e-0c43-49ce-acfd-8ee2587ea255 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:59.523545 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-4c506f6b-58e8-45e7-a4b9-57f4565bc243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:54:59.607018 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 392/762] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:54:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:54:59.607874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c506f6b-58e8-45e7-a4b9-57f4565bc243 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:59.653018 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:59.667219 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:54:59.683945 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a228e-0c43-49ce-acfd-8ee2587ea255 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:59.721165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65b99ec-efa7-4a83-b595-6a3432010622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:59.722121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b99ec-efa7-4a83-b595-6a3432010622 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:59.722607 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b99ec-efa7-4a83-b595-6a3432010622 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:59.723086 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b99ec-efa7-4a83-b595-6a3432010622 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:59.724292 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65b99ec-efa7-4a83-b595-6a3432010622 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:59.724825 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65b99ec-efa7-4a83-b595-6a3432010622 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:59.726530 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65b99ec-efa7-4a83-b595-6a3432010622 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:59.741998 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c506f6b-58e8-45e7-a4b9-57f4565bc243 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:59.752431 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 371/763] 10.222.0.40 () {56 vars in 1149 bytes} [Wed Aug 3 04:54:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:59.753771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65b99ec-efa7-4a83-b595-6a3432010622 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:59.834869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc0115-9290-4880-81a8-3523658f5dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:59.834869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc0115-9290-4880-81a8-3523658f5dfc None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:59.834869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc0115-9290-4880-81a8-3523658f5dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:59.834869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc0115-9290-4880-81a8-3523658f5dfc None admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:59.835298 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc0115-9290-4880-81a8-3523658f5dfc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:54:59.835298 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc0115-9290-4880-81a8-3523658f5dfc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:59.836238 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc0115-9290-4880-81a8-3523658f5dfc None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:54:59.881009 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 393/764] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Aug 3 04:54:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:54:59.882596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc0115-9290-4880-81a8-3523658f5dfc None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:54:59.925541 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65b99ec-efa7-4a83-b595-6a3432010622 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:54:59.992021 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6ff05-0839-481d-b8eb-50602a3adcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:54:59.992021 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6ff05-0839-481d-b8eb-50602a3adcd5 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:54:59.992021 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6ff05-0839-481d-b8eb-50602a3adcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:54:59.992021 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6ff05-0839-481d-b8eb-50602a3adcd5 None admin] PATH_INFO: `/v3/services/network` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:54:59.993197 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6ff05-0839-481d-b8eb-50602a3adcd5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:54:59.994993 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6ff05-0839-481d-b8eb-50602a3adcd5 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:00.018178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-7cb6ff05-0839-481d-b8eb-50602a3adcd5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 04:55:00.018178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:00.018178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 04:55:00.018178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 04:55:00.018178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 04:55:00.018178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 04:55:00.018178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 04:55:00.018178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:00.018178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 04:55:00.018178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:00.018178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:55:00.018178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:00.018178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:00.018178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:00.018178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:00.018178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:00.018178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:00.019295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:00.019295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:00.019295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:00.019295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:00.019295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:00.019295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:00.019295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 04:55:00.019295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 04:55:00.019295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 04:55:00.019295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 04:55:00.019295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:00.019295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:00.019295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:00.019295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:00.019295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:00.019295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:00.020176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:00.020176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:00.020176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:00.020176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:00.020176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:00.020176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:00.020176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:00.020176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:00.020176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:00.020176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:00.020176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 04:55:00.020176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:00.020176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 04:55:00.020176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:00.026088 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 372/765] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:54:59 2022] GET /identity/v3/services/network => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:00.030757 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6ff05-0839-481d-b8eb-50602a3adcd5 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:00.033944 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:00.131177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:00.152233 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc0115-9290-4880-81a8-3523658f5dfc None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:00.276439 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6ff05-0839-481d-b8eb-50602a3adcd5 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:00.289012 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8596dd52-a06b-4c95-9c92-a776cd7ab37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:00.289786 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596dd52-a06b-4c95-9c92-a776cd7ab37a None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:00.290069 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596dd52-a06b-4c95-9c92-a776cd7ab37a None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:00.290342 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596dd52-a06b-4c95-9c92-a776cd7ab37a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:00.335283 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430925b6-9722-4e76-93ef-4a22a0752d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:00.335971 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430925b6-9722-4e76-93ef-4a22a0752d5c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:00.336260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430925b6-9722-4e76-93ef-4a22a0752d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:00.336863 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430925b6-9722-4e76-93ef-4a22a0752d5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:00.351783 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430925b6-9722-4e76-93ef-4a22a0752d5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:00.353427 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430925b6-9722-4e76-93ef-4a22a0752d5c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:00.371036 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-430925b6-9722-4e76-93ef-4a22a0752d5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:00.371036 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:00.371036 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:00.371036 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:00.371036 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:00.371036 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:00.371036 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:00.371036 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:00.371036 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:00.371036 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:00.371036 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:00.371036 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:00.371036 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:55:00.371036 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:55:00.371036 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:55:00.372019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:55:00.372019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:00.372019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:00.372019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:00.372019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:00.372019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:00.372019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:00.372019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:00.372019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:00.372019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:00.372019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:00.372019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:00.372019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:00.372019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:00.372019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:00.372019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:00.372019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:00.373382 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:55:00.373382 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:55:00.373382 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:00.373382 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:00.373382 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 373/766] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:00.373382 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430925b6-9722-4e76-93ef-4a22a0752d5c None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:00.386912 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8596dd52-a06b-4c95-9c92-a776cd7ab37a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:00.389489 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8596dd52-a06b-4c95-9c92-a776cd7ab37a None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:00.442171 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:00.498810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430925b6-9722-4e76-93ef-4a22a0752d5c None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:00.505359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:00.516811 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 394/767] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:54:59 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 634 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 04:55:00.522496 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c2482-9581-4a40-91dc-7fd7c4fbb403 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:00.522784 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c2482-9581-4a40-91dc-7fd7c4fbb403 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:00.523348 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c2482-9581-4a40-91dc-7fd7c4fbb403 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:00.524639 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 395/768] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:00.525381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c2482-9581-4a40-91dc-7fd7c4fbb403 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:00.550226 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62082d-cd2f-497d-8093-83ab7b645e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:00.550883 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62082d-cd2f-497d-8093-83ab7b645e79 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:00.551234 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62082d-cd2f-497d-8093-83ab7b645e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:00.551571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62082d-cd2f-497d-8093-83ab7b645e79 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:00.552236 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d62082d-cd2f-497d-8093-83ab7b645e79 None admin] RBAC: Adding query filter params (name=network) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:00.552526 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d62082d-cd2f-497d-8093-83ab7b645e79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:00.553894 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d62082d-cd2f-497d-8093-83ab7b645e79 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:00.559441 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 374/769] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Aug 3 04:55:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:00.565103 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62082d-cd2f-497d-8093-83ab7b645e79 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:00.691203 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c2482-9581-4a40-91dc-7fd7c4fbb403 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:00.712625 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62082d-cd2f-497d-8093-83ab7b645e79 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:00.730376 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166402d-9b68-4f80-a666-7e6e8b57984a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:00.731031 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166402d-9b68-4f80-a666-7e6e8b57984a None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:00.731315 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166402d-9b68-4f80-a666-7e6e8b57984a None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:00.731589 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166402d-9b68-4f80-a666-7e6e8b57984a None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:00.732278 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166402d-9b68-4f80-a666-7e6e8b57984a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:00.732608 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166402d-9b68-4f80-a666-7e6e8b57984a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:00.733054 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:00.734685 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166402d-9b68-4f80-a666-7e6e8b57984a None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:00.747000 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff54b49e-5778-4be4-a8d2-85ffc008b80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:00.747581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54b49e-5778-4be4-a8d2-85ffc008b80a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:00.747867 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54b49e-5778-4be4-a8d2-85ffc008b80a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:00.748713 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54b49e-5778-4be4-a8d2-85ffc008b80a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:00.754946 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 396/770] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:55:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:00.754946 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d621ec-c459-4811-bc12-781aa9c7879e None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:00.754946 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d621ec-c459-4811-bc12-781aa9c7879e None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:00.754946 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d621ec-c459-4811-bc12-781aa9c7879e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:00.759562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff54b49e-5778-4be4-a8d2-85ffc008b80a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:00.761506 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff54b49e-5778-4be4-a8d2-85ffc008b80a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:00.774566 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-f6d621ec-c459-4811-bc12-781aa9c7879e None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:00.775858 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-f6d621ec-c459-4811-bc12-781aa9c7879e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:00.777062 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-ff54b49e-5778-4be4-a8d2-85ffc008b80a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:00.777062 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:00.777062 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:00.777062 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:00.777062 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:00.777062 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:00.777062 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:00.777062 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:00.777062 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:00.777062 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:00.777062 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:00.777062 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:00.777062 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:55:00.777062 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:55:00.777062 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:55:00.777959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:55:00.777959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:00.777959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:00.777959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:00.777959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:00.777959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:00.777959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:00.777959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:00.777959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:00.777959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:00.777959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:00.777959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:00.777959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:00.777959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:00.777959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:00.777959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:00.777959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:00.779962 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:55:00.779962 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:55:00.779962 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:00.779962 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:00.779962 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 375/771] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:00.779962 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff54b49e-5778-4be4-a8d2-85ffc008b80a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:00.831474 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:00.853776 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 397/772] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:55:00.854692 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d621ec-c459-4811-bc12-781aa9c7879e None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:00.900671 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff54b49e-5778-4be4-a8d2-85ffc008b80a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:00.935100 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08d1e7-6a97-4bdc-af68-3a45b0379c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:00.936071 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08d1e7-6a97-4bdc-af68-3a45b0379c94 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:00.936359 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08d1e7-6a97-4bdc-af68-3a45b0379c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:00.936635 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08d1e7-6a97-4bdc-af68-3a45b0379c94 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:00.937343 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e08d1e7-6a97-4bdc-af68-3a45b0379c94 None admin] RBAC: Adding query filter params (type=network) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:00.937957 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e08d1e7-6a97-4bdc-af68-3a45b0379c94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:00.939683 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e08d1e7-6a97-4bdc-af68-3a45b0379c94 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:00.944980 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 376/773] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Aug 3 04:55:00 2022] GET /identity/v3/services?type=network => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:00.945778 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08d1e7-6a97-4bdc-af68-3a45b0379c94 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:00.966366 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d621ec-c459-4811-bc12-781aa9c7879e None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:00.983777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:01.008210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e069539-a9d2-4313-ae66-162b10a2c816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:01.009013 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e069539-a9d2-4313-ae66-162b10a2c816 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:01.009298 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e069539-a9d2-4313-ae66-162b10a2c816 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:01.009880 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e069539-a9d2-4313-ae66-162b10a2c816 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:01.019091 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e069539-a9d2-4313-ae66-162b10a2c816 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:01.020873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e069539-a9d2-4313-ae66-162b10a2c816 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:01.033217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-0e069539-a9d2-4313-ae66-162b10a2c816 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:55:01.033217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:01.033217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:01.033217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:01.033217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:01.033217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:01.033217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:01.033217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:01.033217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:01.033217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:01.033217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:01.033217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:01.033217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:55:01.033217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:55:01.033217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:55:01.034148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:55:01.034148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:01.034148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:01.034148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:01.034148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:01.034148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:01.034148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:01.034148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:01.034148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:01.034148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:01.034148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:01.034148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:01.034148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:01.034148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:01.034148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:01.034148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:01.034148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:01.035107 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:55:01.035107 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:55:01.035107 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:55:01.035107 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:55:01.035107 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:55:01.035107 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:55:01.035107 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:55:01.035107 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:01.035107 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 398/774] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:55:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:01.036083 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e069539-a9d2-4313-ae66-162b10a2c816 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:01.061455 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08d1e7-6a97-4bdc-af68-3a45b0379c94 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:01.079795 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:01.098147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba64727-b80c-40c7-869d-81cc60ca2864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:01.098937 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba64727-b80c-40c7-869d-81cc60ca2864 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:01.099219 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba64727-b80c-40c7-869d-81cc60ca2864 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:01.099605 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba64727-b80c-40c7-869d-81cc60ca2864 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:01.100317 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba64727-b80c-40c7-869d-81cc60ca2864 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:01.100597 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba64727-b80c-40c7-869d-81cc60ca2864 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:01.101959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba64727-b80c-40c7-869d-81cc60ca2864 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:01.120701 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 377/775] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:55:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:01.120701 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba64727-b80c-40c7-869d-81cc60ca2864 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:01.149089 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e069539-a9d2-4313-ae66-162b10a2c816 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:01.191767 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accffa5e-6d3d-4a1e-90bf-a701fd2eb805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:01.192461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accffa5e-6d3d-4a1e-90bf-a701fd2eb805 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:01.193068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accffa5e-6d3d-4a1e-90bf-a701fd2eb805 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:01.193068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accffa5e-6d3d-4a1e-90bf-a701fd2eb805 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:01.193641 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accffa5e-6d3d-4a1e-90bf-a701fd2eb805 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:01.195062 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accffa5e-6d3d-4a1e-90bf-a701fd2eb805 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:01.204612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-accffa5e-6d3d-4a1e-90bf-a701fd2eb805 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 04:55:01.204612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:01.204612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 04:55:01.204612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 04:55:01.204612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 04:55:01.204612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 04:55:01.204612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 04:55:01.204612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:01.204612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 04:55:01.204612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:01.204612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:55:01.204612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:01.204612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:01.204612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:01.204612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:01.204612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:01.205560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:01.205560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:01.205560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:01.205560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:01.205560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:01.205560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:01.205560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:01.205560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 04:55:01.205560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 04:55:01.205560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 04:55:01.205560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 04:55:01.205560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:01.205560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:01.205560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:01.205560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:01.205560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:01.206411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:01.206411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:01.206411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:01.206411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:01.206411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:01.206411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:01.206411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:01.206411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:01.206411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:01.206411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:01.206411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:01.206411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 04:55:01.206411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:01.206411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 04:55:01.206411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:01.206411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 399/776] 10.222.0.40 () {56 vars in 1129 bytes} [Wed Aug 3 04:55:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:01.208054 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accffa5e-6d3d-4a1e-90bf-a701fd2eb805 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:01.231318 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:01.238791 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba64727-b80c-40c7-869d-81cc60ca2864 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:01.281059 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd1047-6f02-42ff-ad14-dda61915d6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:01.281842 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd1047-6f02-42ff-ad14-dda61915d6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:01.282138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd1047-6f02-42ff-ad14-dda61915d6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:01.283195 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd1047-6f02-42ff-ad14-dda61915d6b5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:01.283954 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd1047-6f02-42ff-ad14-dda61915d6b5 None admin] RBAC: Adding query filter params (interface=public, service_id=226246c45b8d41d1a426b73cd96a0997, region_id=RegionOne) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:01.284268 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd1047-6f02-42ff-ad14-dda61915d6b5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:01.285638 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd1047-6f02-42ff-ad14-dda61915d6b5 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:01.301534 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 378/777] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Aug 3 04:55:01 2022] GET /identity/v3/endpoints?service_id=226246c45b8d41d1a426b73cd96a0997&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:01.301534 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd1047-6f02-42ff-ad14-dda61915d6b5 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:01.312595 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accffa5e-6d3d-4a1e-90bf-a701fd2eb805 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:01.315048 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:01.348114 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35f83e-3843-4fc5-9d6c-4c3db0a7bd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:01.348728 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35f83e-3843-4fc5-9d6c-4c3db0a7bd65 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:01.349008 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35f83e-3843-4fc5-9d6c-4c3db0a7bd65 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:01.349280 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35f83e-3843-4fc5-9d6c-4c3db0a7bd65 None admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:01.350114 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f35f83e-3843-4fc5-9d6c-4c3db0a7bd65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:01.350416 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f35f83e-3843-4fc5-9d6c-4c3db0a7bd65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:01.351876 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f35f83e-3843-4fc5-9d6c-4c3db0a7bd65 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:01.364509 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 400/778] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:55:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:01.365326 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35f83e-3843-4fc5-9d6c-4c3db0a7bd65 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:01.455699 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd1047-6f02-42ff-ad14-dda61915d6b5 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:01.471665 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35f83e-3843-4fc5-9d6c-4c3db0a7bd65 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:01.489656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261b5c6-1de7-4de9-89c1-ac7af4a76db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:01.490297 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261b5c6-1de7-4de9-89c1-ac7af4a76db3 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:01.490580 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261b5c6-1de7-4de9-89c1-ac7af4a76db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:01.490882 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261b5c6-1de7-4de9-89c1-ac7af4a76db3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:01.497446 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261b5c6-1de7-4de9-89c1-ac7af4a76db3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:01.498972 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261b5c6-1de7-4de9-89c1-ac7af4a76db3 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:01.506450 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e54f50-c0b1-4531-b287-5101c826e080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:01.507100 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e54f50-c0b1-4531-b287-5101c826e080 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:01.507388 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e54f50-c0b1-4531-b287-5101c826e080 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:01.507667 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e54f50-c0b1-4531-b287-5101c826e080 None admin] PATH_INFO: `/v3/services` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:01.508337 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e54f50-c0b1-4531-b287-5101c826e080 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:01.508645 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e54f50-c0b1-4531-b287-5101c826e080 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:01.510019 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e54f50-c0b1-4531-b287-5101c826e080 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:01.510169 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-9261b5c6-1de7-4de9-89c1-ac7af4a76db3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:55:01.510169 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:01.510169 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:01.510169 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:01.510169 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:01.510169 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:01.510169 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:01.510169 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:01.510169 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:01.510169 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:01.510169 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:01.510169 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:01.510169 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:55:01.510169 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:55:01.510169 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:55:01.511166 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:55:01.511166 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:01.511166 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:01.511166 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:01.511166 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:01.511166 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:01.511166 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:01.511166 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:01.511166 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:01.511166 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:01.511166 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:01.511166 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:01.511166 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:01.511166 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:01.511166 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:01.511166 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:01.511166 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:01.512085 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:55:01.512085 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:55:01.512085 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:55:01.512085 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:55:01.512085 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:55:01.512085 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:55:01.512085 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:55:01.512085 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:01.512085 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 379/779] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:55:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:01.512085 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261b5c6-1de7-4de9-89c1-ac7af4a76db3 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:01.519423 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 401/780] 10.222.0.40 () {56 vars in 1124 bytes} [Wed Aug 3 04:55:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:01.520419 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e54f50-c0b1-4531-b287-5101c826e080 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:01.535098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:01.552490 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:01.613209 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261b5c6-1de7-4de9-89c1-ac7af4a76db3 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:01.622972 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e54f50-c0b1-4531-b287-5101c826e080 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:01.651238 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a9656-02d3-4626-ad24-f415f5631b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:01.651848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a9656-02d3-4626-ad24-f415f5631b57 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:01.652135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a9656-02d3-4626-ad24-f415f5631b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:01.652415 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a9656-02d3-4626-ad24-f415f5631b57 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:01.653071 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a9656-02d3-4626-ad24-f415f5631b57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:01.654957 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a9656-02d3-4626-ad24-f415f5631b57 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:01.659322 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfaf38d-36fa-4287-8cf1-07a8107ae476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:01.670975 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfaf38d-36fa-4287-8cf1-07a8107ae476 None admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:01.671740 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 380/781] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Aug 3 04:55:01 2022] GET /identity/v3/services => generated 772 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:01.671823 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfaf38d-36fa-4287-8cf1-07a8107ae476 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:01.671823 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfaf38d-36fa-4287-8cf1-07a8107ae476 None admin] PATH_INFO: `/v3/projects/cf3763a37383456184a9ca9351b4ce03/users/a3fa4a6c16274c18849869b55b33ac62/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:01.671980 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf63737-c3a8-408d-b48e-2f1e33736c34 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:01.671980 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf63737-c3a8-408d-b48e-2f1e33736c34 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:01.671980 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf63737-c3a8-408d-b48e-2f1e33736c34 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:01.671980 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 381/782] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:01.671980 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf63737-c3a8-408d-b48e-2f1e33736c34 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:01.672360 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfaf38d-36fa-4287-8cf1-07a8107ae476 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf3763a37383456184a9ca9351b4ce03, user_id=a3fa4a6c16274c18849869b55b33ac62, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:01.672360 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfaf38d-36fa-4287-8cf1-07a8107ae476 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:01.746076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 402/783] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 3 04:55:01 2022] PUT /identity/v3/projects/cf3763a37383456184a9ca9351b4ce03/users/a3fa4a6c16274c18849869b55b33ac62/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 04:55:01.751931 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfaf38d-36fa-4287-8cf1-07a8107ae476 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:01.765006 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:01.796184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf63737-c3a8-408d-b48e-2f1e33736c34 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:01.856657 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c1e92-c641-41da-92d0-2e0c77c263f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:01.858656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c1e92-c641-41da-92d0-2e0c77c263f0 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:01.859313 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c1e92-c641-41da-92d0-2e0c77c263f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:01.859599 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c1e92-c641-41da-92d0-2e0c77c263f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:01.860378 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948c1e92-c641-41da-92d0-2e0c77c263f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:01.860689 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948c1e92-c641-41da-92d0-2e0c77c263f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:01.862171 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948c1e92-c641-41da-92d0-2e0c77c263f0 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:01.880879 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 382/784] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:55:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:01.880879 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c532d93-80e7-4410-aa6c-6f2db85b0f4c None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:01.880879 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c532d93-80e7-4410-aa6c-6f2db85b0f4c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:01.880879 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c532d93-80e7-4410-aa6c-6f2db85b0f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:01.899135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-0c532d93-80e7-4410-aa6c-6f2db85b0f4c None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:01.900507 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0c532d93-80e7-4410-aa6c-6f2db85b0f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:01.909843 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:01.934241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfaf38d-36fa-4287-8cf1-07a8107ae476 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:01.982181 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c1d4f-1aaf-453d-8f52-470cd8353dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:01.983483 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 383/785] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:55:01.983964 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c1d4f-1aaf-453d-8f52-470cd8353dab None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:01.984494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c1d4f-1aaf-453d-8f52-470cd8353dab None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:01.984620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c532d93-80e7-4410-aa6c-6f2db85b0f4c None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:01.985075 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c1d4f-1aaf-453d-8f52-470cd8353dab None admin] PATH_INFO: `/v3/services` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:01.985927 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c1d4f-1aaf-453d-8f52-470cd8353dab None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:01.986409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c1d4f-1aaf-453d-8f52-470cd8353dab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:01.989547 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c1d4f-1aaf-453d-8f52-470cd8353dab None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:02.000953 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 403/786] 10.222.0.40 () {56 vars in 1124 bytes} [Wed Aug 3 04:55:01 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:02.004298 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c1d4f-1aaf-453d-8f52-470cd8353dab None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:02.069925 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:02.097703 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c532d93-80e7-4410-aa6c-6f2db85b0f4c None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:02.140601 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19038e37-e91e-4e8e-888e-36e88e2c9c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:02.141215 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19038e37-e91e-4e8e-888e-36e88e2c9c55 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:02.141494 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19038e37-e91e-4e8e-888e-36e88e2c9c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:02.141771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19038e37-e91e-4e8e-888e-36e88e2c9c55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:02.142551 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19038e37-e91e-4e8e-888e-36e88e2c9c55 None admin] RBAC: Adding query filter params (role.id=cae571342db14f5fac7849efe7f8636e, scope.project.id=cf3763a37383456184a9ca9351b4ce03, user.id=de853a2dfffb4f05b6f47df2280ae4e1) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:02.142876 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19038e37-e91e-4e8e-888e-36e88e2c9c55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:02.144309 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19038e37-e91e-4e8e-888e-36e88e2c9c55 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:02.150369 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 384/787] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:55:01 2022] GET /identity/v3/role_assignments?user.id=de853a2dfffb4f05b6f47df2280ae4e1&scope.project.id=cf3763a37383456184a9ca9351b4ce03&role.id=cae571342db14f5fac7849efe7f8636e => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:02.151195 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19038e37-e91e-4e8e-888e-36e88e2c9c55 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:02.151748 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c1d4f-1aaf-453d-8f52-470cd8353dab None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:02.203454 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecee08d-c9f5-4ed7-9d93-94b90d10f1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:02.204390 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecee08d-c9f5-4ed7-9d93-94b90d10f1e0 None admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:02.204681 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecee08d-c9f5-4ed7-9d93-94b90d10f1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:02.204956 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecee08d-c9f5-4ed7-9d93-94b90d10f1e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:02.205954 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecee08d-c9f5-4ed7-9d93-94b90d10f1e0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:02.208261 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecee08d-c9f5-4ed7-9d93-94b90d10f1e0 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:02.296136 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19038e37-e91e-4e8e-888e-36e88e2c9c55 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:02.335002 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7205a9d-14db-4ad1-9b3e-982dce1ef65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:02.335705 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7205a9d-14db-4ad1-9b3e-982dce1ef65b None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:02.335982 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7205a9d-14db-4ad1-9b3e-982dce1ef65b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:02.336271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7205a9d-14db-4ad1-9b3e-982dce1ef65b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:02.344266 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:02.349213 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7205a9d-14db-4ad1-9b3e-982dce1ef65b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:55:02.352613 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 404/788] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 04:55:02 2022] POST /identity/v3/endpoints => generated 346 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:55:02.354192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-c7205a9d-14db-4ad1-9b3e-982dce1ef65b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:55:02.354192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:02.354192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:02.354192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:02.354192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:02.354192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:02.354192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:02.354192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:02.354192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:02.354192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:02.354192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:02.354192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:02.354192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:55:02.354192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:55:02.354192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:55:02.354192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:55:02.355160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:55:02.355160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:55:02.355160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:55:02.355160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:55:02.355160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:02.355160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:02.355160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:02.355160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:02.355160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:02.355160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:02.355160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:02.355160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:02.355160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:02.355160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:02.355160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:02.355160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:02.356610 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:02.356610 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:02.356610 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:02.356610 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:02.356610 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:55:02.356610 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:55:02.356610 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:55:02.356610 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:55:02.356610 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:55:02.356610 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:55:02.356610 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:55:02.356610 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:02.356610 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 385/789] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:55:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:02.361842 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecee08d-c9f5-4ed7-9d93-94b90d10f1e0 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:02.375195 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:02.540448 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecee08d-c9f5-4ed7-9d93-94b90d10f1e0 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:02.584021 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84679c9c-8cb2-4a73-86d0-9fe9c1eff03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:02.584570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84679c9c-8cb2-4a73-86d0-9fe9c1eff03c None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:02.584850 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84679c9c-8cb2-4a73-86d0-9fe9c1eff03c None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:02.585123 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84679c9c-8cb2-4a73-86d0-9fe9c1eff03c None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:02.585842 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84679c9c-8cb2-4a73-86d0-9fe9c1eff03c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:02.586143 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84679c9c-8cb2-4a73-86d0-9fe9c1eff03c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:02.587552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84679c9c-8cb2-4a73-86d0-9fe9c1eff03c None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:02.597338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 405/790] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:55:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:02.602969 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7205a9d-14db-4ad1-9b3e-982dce1ef65b None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:02.725744 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7205a9d-14db-4ad1-9b3e-982dce1ef65b None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:02.761607 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401478f1-3b33-45c0-aa93-91643c7c5059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:02.762242 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401478f1-3b33-45c0-aa93-91643c7c5059 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:02.762524 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401478f1-3b33-45c0-aa93-91643c7c5059 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:02.762793 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401478f1-3b33-45c0-aa93-91643c7c5059 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:02.774574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401478f1-3b33-45c0-aa93-91643c7c5059 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:02.776127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401478f1-3b33-45c0-aa93-91643c7c5059 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:02.795796 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-401478f1-3b33-45c0-aa93-91643c7c5059 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:02.795796 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:02.795796 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:02.795796 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:02.795796 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:02.795796 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:02.795796 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:02.795796 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:02.795796 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:02.795796 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:02.795796 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:02.795796 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:02.795796 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:55:02.795796 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:55:02.795796 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:55:02.796827 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:55:02.796827 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:02.796827 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:02.796827 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:02.796827 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:02.796827 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:02.796827 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:02.796827 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:02.796827 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:02.796827 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:02.796827 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:02.796827 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:02.796827 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:02.796827 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:02.796827 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:02.796827 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:02.796827 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:02.798317 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:55:02.798317 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:55:02.798317 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:02.798317 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:02.798317 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 386/791] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:02.802833 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84679c9c-8cb2-4a73-86d0-9fe9c1eff03c None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:02.879467 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:02.928684 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84679c9c-8cb2-4a73-86d0-9fe9c1eff03c None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:02.979610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78238e1e-861a-4d7a-8083-96839c6ff00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:02.980231 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78238e1e-861a-4d7a-8083-96839c6ff00e None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:02.980510 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78238e1e-861a-4d7a-8083-96839c6ff00e None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:02.980783 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78238e1e-861a-4d7a-8083-96839c6ff00e None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:02.981473 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78238e1e-861a-4d7a-8083-96839c6ff00e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:02.981778 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78238e1e-861a-4d7a-8083-96839c6ff00e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:02.983327 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78238e1e-861a-4d7a-8083-96839c6ff00e None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:02.995697 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 406/792] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:55:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:03.001900 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401478f1-3b33-45c0-aa93-91643c7c5059 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:03.069910 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:03.133474 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401478f1-3b33-45c0-aa93-91643c7c5059 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:03.179531 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3f264c-eceb-4b7d-b7a3-0d22aec1e17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:03.180539 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f264c-eceb-4b7d-b7a3-0d22aec1e17f None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:03.181034 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f264c-eceb-4b7d-b7a3-0d22aec1e17f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:03.181536 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f264c-eceb-4b7d-b7a3-0d22aec1e17f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:03.196918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3f264c-eceb-4b7d-b7a3-0d22aec1e17f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:03.198820 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3f264c-eceb-4b7d-b7a3-0d22aec1e17f None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:03.218799 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-ce3f264c-eceb-4b7d-b7a3-0d22aec1e17f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 03 04:55:03.218799 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:03.218799 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:03.218799 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:03.218799 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:03.218799 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:03.218799 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:03.218799 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:03.218799 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:03.218799 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:03.218799 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:03.218799 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:03.218799 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:55:03.218799 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:55:03.218799 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:55:03.218799 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:55:03.219828 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:03.219828 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:03.219828 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:55:03.219828 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:55:03.219828 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:55:03.219828 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:55:03.219828 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:03.219828 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:03.219828 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:03.219828 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:03.219828 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:03.219828 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:03.219828 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:03.219828 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:03.219828 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:03.219828 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:03.219828 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:03.219828 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:03.220776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:03.220776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:03.220776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:55:03.220776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:55:03.220776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:55:03.220776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:55:03.220776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:55:03.220776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:55:03.220776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 03 04:55:03.220776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:03.223093 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 387/793] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:55:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:03.228833 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78238e1e-861a-4d7a-8083-96839c6ff00e None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:03.375362 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78238e1e-861a-4d7a-8083-96839c6ff00e None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:03.409798 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:03.418241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f161ab-15ca-4263-856a-377a490e6184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:03.422483 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f161ab-15ca-4263-856a-377a490e6184 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:03.422483 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f161ab-15ca-4263-856a-377a490e6184 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:03.422483 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f161ab-15ca-4263-856a-377a490e6184 None admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:03.422483 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f161ab-15ca-4263-856a-377a490e6184 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:03.422483 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f161ab-15ca-4263-856a-377a490e6184 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:03.422483 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f161ab-15ca-4263-856a-377a490e6184 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:03.445321 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 407/794] 10.222.0.40 () {56 vars in 1135 bytes} [Wed Aug 3 04:55:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:03.451548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3f264c-eceb-4b7d-b7a3-0d22aec1e17f None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:03.595562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3f264c-eceb-4b7d-b7a3-0d22aec1e17f None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:03.609525 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:03.637620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479beac-cf41-4b34-bbe8-a68a4a40e30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:03.638269 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479beac-cf41-4b34-bbe8-a68a4a40e30d None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:03.639037 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479beac-cf41-4b34-bbe8-a68a4a40e30d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:03.639319 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479beac-cf41-4b34-bbe8-a68a4a40e30d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:03.651516 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479beac-cf41-4b34-bbe8-a68a4a40e30d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:03.653203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479beac-cf41-4b34-bbe8-a68a4a40e30d None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:03.669135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-4479beac-cf41-4b34-bbe8-a68a4a40e30d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:03.669135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:03.669135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:03.669135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:03.669135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:03.669135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:03.669135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:03.669135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:03.669135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:03.669135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:03.669135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:03.669135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:03.669135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:55:03.669135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:55:03.669135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:55:03.670857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:55:03.670857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:03.670857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:03.670857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:03.670857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:03.670857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:03.670857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:03.670857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:03.670857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:03.670857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:03.670857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:03.670857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:03.670857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:03.670857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:03.670857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:03.670857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:03.670857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:03.672026 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:55:03.672026 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:55:03.672026 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:03.672026 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:03.672702 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 388/795] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:03.680520 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f161ab-15ca-4263-856a-377a490e6184 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:03.829622 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f161ab-15ca-4263-856a-377a490e6184 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:03.876029 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d3e2b-3266-4510-92c5-d6c8c47524f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:03.876650 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d3e2b-3266-4510-92c5-d6c8c47524f4 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:03.876929 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d3e2b-3266-4510-92c5-d6c8c47524f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:03.877200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d3e2b-3266-4510-92c5-d6c8c47524f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:03.877890 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d3e2b-3266-4510-92c5-d6c8c47524f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:03.878191 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d3e2b-3266-4510-92c5-d6c8c47524f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:03.879667 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d3e2b-3266-4510-92c5-d6c8c47524f4 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:03.898364 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 408/796] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:03.906055 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479beac-cf41-4b34-bbe8-a68a4a40e30d None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:04.046803 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479beac-cf41-4b34-bbe8-a68a4a40e30d None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:04.094984 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97280b6-4c25-40de-b836-2704bec93804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:04.095639 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97280b6-4c25-40de-b836-2704bec93804 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:04.095925 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97280b6-4c25-40de-b836-2704bec93804 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:04.096203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97280b6-4c25-40de-b836-2704bec93804 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:04.103529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97280b6-4c25-40de-b836-2704bec93804 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:04.105007 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97280b6-4c25-40de-b836-2704bec93804 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:04.122965 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-b97280b6-4c25-40de-b836-2704bec93804 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:55:04.122965 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:04.122965 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:04.122965 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:04.122965 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:04.122965 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:04.122965 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:04.122965 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:04.122965 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:04.122965 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:04.122965 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:04.122965 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:04.122965 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:55:04.122965 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:55:04.122965 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:55:04.124632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:55:04.124632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:04.124632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:04.124632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:04.124632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:04.124632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:04.124632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:04.124632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:04.124632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:04.124632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:04.124632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:04.124632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:04.124632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:04.124632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:04.124632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:04.124632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:04.124632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:04.125615 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:55:04.125615 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:55:04.125615 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:55:04.125615 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:55:04.125615 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:55:04.125615 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:55:04.125615 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:55:04.125615 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:04.126620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 389/797] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:55:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:04.137567 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d3e2b-3266-4510-92c5-d6c8c47524f4 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:04.170538 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deba40d-0cfe-4e86-ab50-ce38113eafd6 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:04.170881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deba40d-0cfe-4e86-ab50-ce38113eafd6 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:04.171172 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deba40d-0cfe-4e86-ab50-ce38113eafd6 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:04.172512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 390/798] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:55:04.177874 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e01dfe-b4d3-4356-afe9-ef24822e7c82 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:04.178162 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e01dfe-b4d3-4356-afe9-ef24822e7c82 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:04.178441 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e01dfe-b4d3-4356-afe9-ef24822e7c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:04.202979 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-97e01dfe-b4d3-4356-afe9-ef24822e7c82 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:04.204934 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-97e01dfe-b4d3-4356-afe9-ef24822e7c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:04.220005 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:04.229399 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:04.291886 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d3e2b-3266-4510-92c5-d6c8c47524f4 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:04.295653 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 391/799] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:55:04.296996 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db39f77-a678-4d29-b887-ffa907e427da None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:04.297223 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db39f77-a678-4d29-b887-ffa907e427da None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:04.297612 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db39f77-a678-4d29-b887-ffa907e427da None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:04.302911 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 392/800] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:04.308571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8dd4e2-ce7d-428e-b262-8e0ebeeecdfd None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:04.308571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8dd4e2-ce7d-428e-b262-8e0ebeeecdfd None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:04.308571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8dd4e2-ce7d-428e-b262-8e0ebeeecdfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:04.324721 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-df8dd4e2-ce7d-428e-b262-8e0ebeeecdfd None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:04.325904 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-df8dd4e2-ce7d-428e-b262-8e0ebeeecdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:04.353801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5d085-eec6-4a55-88fb-4aa45641917e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:04.354739 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5d085-eec6-4a55-88fb-4aa45641917e None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:04.355181 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5d085-eec6-4a55-88fb-4aa45641917e None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:04.355874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5d085-eec6-4a55-88fb-4aa45641917e None admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:04.357018 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5d085-eec6-4a55-88fb-4aa45641917e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:04.357461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5d085-eec6-4a55-88fb-4aa45641917e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:04.359773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5d085-eec6-4a55-88fb-4aa45641917e None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:04.376006 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 409/801] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:55:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:04.377045 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5d085-eec6-4a55-88fb-4aa45641917e None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:04.409339 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 393/802] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:55:04.415790 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8dd4e2-ce7d-428e-b262-8e0ebeeecdfd None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:04.486387 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5d085-eec6-4a55-88fb-4aa45641917e None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:04.489040 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:04.525020 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be47cc-5c88-4d46-8543-ea9396e3e818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:04.525670 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be47cc-5c88-4d46-8543-ea9396e3e818 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:04.525954 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be47cc-5c88-4d46-8543-ea9396e3e818 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:04.526239 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be47cc-5c88-4d46-8543-ea9396e3e818 None admin] PATH_INFO: `/v3/services/network` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:04.526892 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36be47cc-5c88-4d46-8543-ea9396e3e818 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:04.528299 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:04.529498 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36be47cc-5c88-4d46-8543-ea9396e3e818 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:04.539660 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-36be47cc-5c88-4d46-8543-ea9396e3e818 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 04:55:04.539660 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:04.539660 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 04:55:04.539660 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 04:55:04.539660 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 04:55:04.539660 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 04:55:04.539660 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 04:55:04.539660 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:04.539660 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 04:55:04.539660 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:04.539660 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:55:04.539660 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:04.539660 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:04.539660 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:04.539660 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:04.539660 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:04.539660 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:04.540938 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:04.540938 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:04.540938 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:04.540938 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:04.540938 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:04.540938 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:04.540938 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 04:55:04.540938 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 04:55:04.540938 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 04:55:04.540938 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 04:55:04.540938 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:04.540938 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:04.540938 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:04.540938 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:04.540938 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:04.540938 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:04.541948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:04.541948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:04.541948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:04.541948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:04.541948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:04.541948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:04.541948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:04.541948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:04.541948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:04.541948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:04.541948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 04:55:04.541948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:04.541948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 04:55:04.541948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:04.544328 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 410/803] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:55:04 2022] GET /identity/v3/services/network => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:04.549647 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8dd4e2-ce7d-428e-b262-8e0ebeeecdfd None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:04.550333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be47cc-5c88-4d46-8543-ea9396e3e818 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:04.588899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae4348-8d27-46a7-9122-7df71abf73c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:04.589576 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae4348-8d27-46a7-9122-7df71abf73c9 None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:04.589886 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae4348-8d27-46a7-9122-7df71abf73c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:04.590821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae4348-8d27-46a7-9122-7df71abf73c9 None admin] PATH_INFO: `/v3/projects/cf3763a37383456184a9ca9351b4ce03/users/e1014bcb4d36441aa46042941c9a63d0/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:04.592826 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae4348-8d27-46a7-9122-7df71abf73c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf3763a37383456184a9ca9351b4ce03, user_id=e1014bcb4d36441aa46042941c9a63d0, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:04.594335 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae4348-8d27-46a7-9122-7df71abf73c9 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:04.667031 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 394/804] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 3 04:55:04 2022] PUT /identity/v3/projects/cf3763a37383456184a9ca9351b4ce03/users/e1014bcb4d36441aa46042941c9a63d0/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 04:55:04.668223 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae4348-8d27-46a7-9122-7df71abf73c9 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:04.686415 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be47cc-5c88-4d46-8543-ea9396e3e818 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:04.732782 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876f2c3-c48e-4640-85a7-5ed9713c4440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:04.733903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876f2c3-c48e-4640-85a7-5ed9713c4440 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:04.734392 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876f2c3-c48e-4640-85a7-5ed9713c4440 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:04.734996 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876f2c3-c48e-4640-85a7-5ed9713c4440 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:04.749514 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876f2c3-c48e-4640-85a7-5ed9713c4440 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:55:04.757094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-d876f2c3-c48e-4640-85a7-5ed9713c4440 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:55:04.757094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:04.757094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:04.757094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:04.757094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:04.757094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:04.757094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:04.757094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:04.757094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:04.757094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:04.757094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:04.757094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:04.757094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:55:04.757094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:55:04.757094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:55:04.757094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:55:04.758127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:55:04.758127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:55:04.758127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:55:04.758127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:55:04.758127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:04.758127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:04.758127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:04.758127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:04.758127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:04.758127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:04.758127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:04.758127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:04.758127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:04.758127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:04.758127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:04.758127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:04.759002 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:04.759002 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:04.759002 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:04.759002 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:04.759002 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:55:04.759002 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:55:04.759002 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:55:04.759002 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:55:04.759002 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:55:04.759002 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:55:04.759002 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:55:04.759002 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:04.761704 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 411/805] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:55:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:04.771946 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876f2c3-c48e-4640-85a7-5ed9713c4440 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:04.809126 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:04.900919 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae4348-8d27-46a7-9122-7df71abf73c9 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:04.905754 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876f2c3-c48e-4640-85a7-5ed9713c4440 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:04.941905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9786ab2-f96a-4c2d-b903-ae86697a415c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:04.943318 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9786ab2-f96a-4c2d-b903-ae86697a415c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:04.943849 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9786ab2-f96a-4c2d-b903-ae86697a415c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:04.944372 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9786ab2-f96a-4c2d-b903-ae86697a415c None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:04.944959 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bfc4b-6f62-40c3-af4f-40cb83051177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:04.945408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9786ab2-f96a-4c2d-b903-ae86697a415c None admin] RBAC: Adding query filter params (name=network) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:04.945783 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bfc4b-6f62-40c3-af4f-40cb83051177 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:04.946042 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9786ab2-f96a-4c2d-b903-ae86697a415c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:04.946253 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bfc4b-6f62-40c3-af4f-40cb83051177 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:04.946635 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bfc4b-6f62-40c3-af4f-40cb83051177 None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:04.947502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941bfc4b-6f62-40c3-af4f-40cb83051177 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:04.947979 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941bfc4b-6f62-40c3-af4f-40cb83051177 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:04.948250 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9786ab2-f96a-4c2d-b903-ae86697a415c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:04.949639 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941bfc4b-6f62-40c3-af4f-40cb83051177 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:04.959397 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 395/806] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Aug 3 04:55:04 2022] GET /identity/v3/services?name=network => generated 124 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:04.961516 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de9ef1-a69f-4697-8201-89989bf24a59 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:04.961805 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de9ef1-a69f-4697-8201-89989bf24a59 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:04.962083 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de9ef1-a69f-4697-8201-89989bf24a59 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:04.965274 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 412/808] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:55:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:04.966409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bfc4b-6f62-40c3-af4f-40cb83051177 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:04.968156 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 396/808] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:04.969838 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d0f8d-0894-4938-84a4-6b1e7a9aa5bf None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:04.970411 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d0f8d-0894-4938-84a4-6b1e7a9aa5bf None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:04.970700 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d0f8d-0894-4938-84a4-6b1e7a9aa5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:04.989353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-954d0f8d-0894-4938-84a4-6b1e7a9aa5bf None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:04.990569 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-954d0f8d-0894-4938-84a4-6b1e7a9aa5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:04.991149 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:05.063273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 397/809] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:55:05.064042 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d0f8d-0894-4938-84a4-6b1e7a9aa5bf None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:05.078255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:05.089033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bfc4b-6f62-40c3-af4f-40cb83051177 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:05.124590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54642e-8830-474e-a042-dd54bc272755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:05.125224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54642e-8830-474e-a042-dd54bc272755 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:05.125509 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54642e-8830-474e-a042-dd54bc272755 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:05.125791 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54642e-8830-474e-a042-dd54bc272755 None admin] PATH_INFO: `/v3/services` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:05.126471 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54642e-8830-474e-a042-dd54bc272755 None admin] RBAC: Adding query filter params (type=network) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:05.126781 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54642e-8830-474e-a042-dd54bc272755 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:05.128183 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54642e-8830-474e-a042-dd54bc272755 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:05.137908 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 413/810] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Aug 3 04:55:04 2022] GET /identity/v3/services?type=network => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:05.137908 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27090ab0-9b8e-4f83-942d-75418a2d28b6 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:05.137908 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27090ab0-9b8e-4f83-942d-75418a2d28b6 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:05.137908 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27090ab0-9b8e-4f83-942d-75418a2d28b6 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:05.137908 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 414/811] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:05.137908 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27090ab0-9b8e-4f83-942d-75418a2d28b6 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:05.178672 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d0f8d-0894-4938-84a4-6b1e7a9aa5bf None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:05.215031 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81cb5af-e4cb-4e8e-bf3d-e0efe2cdacf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:05.215676 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81cb5af-e4cb-4e8e-bf3d-e0efe2cdacf8 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:05.215959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81cb5af-e4cb-4e8e-bf3d-e0efe2cdacf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:05.216237 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81cb5af-e4cb-4e8e-bf3d-e0efe2cdacf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:05.219056 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:05.227855 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81cb5af-e4cb-4e8e-bf3d-e0efe2cdacf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:05.229438 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81cb5af-e4cb-4e8e-bf3d-e0efe2cdacf8 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:05.245848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-e81cb5af-e4cb-4e8e-bf3d-e0efe2cdacf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:05.245848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:05.245848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:05.245848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:05.245848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:05.245848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:05.245848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:05.245848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:05.245848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:05.245848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:05.245848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:05.245848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:05.245848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:55:05.245848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:55:05.245848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:55:05.246810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:55:05.246810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:05.246810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:05.246810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:05.246810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:05.246810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:05.246810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:05.246810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:05.246810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:05.246810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:05.246810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:05.246810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:05.246810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:05.246810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:05.246810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:05.246810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:05.246810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:05.248423 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:55:05.248423 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:55:05.248423 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:05.248423 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:05.249900 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 398/812] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:05.250808 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81cb5af-e4cb-4e8e-bf3d-e0efe2cdacf8 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:05.252816 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27090ab0-9b8e-4f83-942d-75418a2d28b6 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:05.291993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d80f6-c1dd-49ff-8a71-3ca95934240f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:05.292643 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d80f6-c1dd-49ff-8a71-3ca95934240f None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:05.292927 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d80f6-c1dd-49ff-8a71-3ca95934240f None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:05.293207 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d80f6-c1dd-49ff-8a71-3ca95934240f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:05.304753 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85d80f6-c1dd-49ff-8a71-3ca95934240f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:55:05.310014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-a85d80f6-c1dd-49ff-8a71-3ca95934240f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:55:05.310014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:05.310014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:05.310014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:05.310014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:05.310014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:05.310014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:05.310014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:05.310014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:05.310014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:05.310014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:05.310014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:05.310014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:55:05.310014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:55:05.310014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:55:05.310014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:55:05.311651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:55:05.311651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:55:05.311651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:55:05.311651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:55:05.311651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:05.311651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:05.311651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:05.311651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:05.311651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:05.311651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:05.311651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:05.311651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:05.311651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:05.311651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:05.311651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:05.311651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:05.312519 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:05.312519 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:05.312519 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:05.312519 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:05.312519 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:55:05.312519 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:55:05.312519 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:55:05.312519 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:55:05.312519 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:55:05.312519 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:55:05.312519 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:55:05.312519 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:05.312519 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 415/813] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 3 04:55:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:05.313236 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c18ad-9a2a-43f1-a822-87059ef32271 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:05.313236 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c18ad-9a2a-43f1-a822-87059ef32271 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:05.313413 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c18ad-9a2a-43f1-a822-87059ef32271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:05.332214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-c65c18ad-9a2a-43f1-a822-87059ef32271 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:05.333437 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-c65c18ad-9a2a-43f1-a822-87059ef32271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:05.343959 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:05.366654 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81cb5af-e4cb-4e8e-bf3d-e0efe2cdacf8 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:05.401842 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05dbc94-3780-42dd-8989-f70d0d3e4f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:05.402465 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05dbc94-3780-42dd-8989-f70d0d3e4f4d None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:05.402747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05dbc94-3780-42dd-8989-f70d0d3e4f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:05.403056 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05dbc94-3780-42dd-8989-f70d0d3e4f4d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:05.403760 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05dbc94-3780-42dd-8989-f70d0d3e4f4d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:05.405003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 416/814] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:55:05.405760 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c18ad-9a2a-43f1-a822-87059ef32271 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:05.411250 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05dbc94-3780-42dd-8989-f70d0d3e4f4d None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:05.493740 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 399/815] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 04:55:05 2022] POST /identity/v3/endpoints => generated 337 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:55:05.494979 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05dbc94-3780-42dd-8989-f70d0d3e4f4d None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:05.526259 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c18ad-9a2a-43f1-a822-87059ef32271 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:05.564298 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:05.596706 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f00e4e7-ea22-4f10-9937-5a5ac15d9545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:05.598699 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00e4e7-ea22-4f10-9937-5a5ac15d9545 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:05.599174 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00e4e7-ea22-4f10-9937-5a5ac15d9545 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:05.599458 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00e4e7-ea22-4f10-9937-5a5ac15d9545 None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:05.600191 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f00e4e7-ea22-4f10-9937-5a5ac15d9545 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:05.600504 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f00e4e7-ea22-4f10-9937-5a5ac15d9545 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:05.603235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f00e4e7-ea22-4f10-9937-5a5ac15d9545 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:05.616241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:05.623509 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 417/816] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:55:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:05.624758 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f00e4e7-ea22-4f10-9937-5a5ac15d9545 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:05.637206 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05dbc94-3780-42dd-8989-f70d0d3e4f4d None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:05.689813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b8fe8-c14e-4843-90c3-c206cd906a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:05.690678 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b8fe8-c14e-4843-90c3-c206cd906a55 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:05.691142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b8fe8-c14e-4843-90c3-c206cd906a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:05.691584 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b8fe8-c14e-4843-90c3-c206cd906a55 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:05.695089 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b8fe8-c14e-4843-90c3-c206cd906a55 None admin] RBAC: Adding query filter params (name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:05.695089 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b8fe8-c14e-4843-90c3-c206cd906a55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:05.695089 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b8fe8-c14e-4843-90c3-c206cd906a55 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:05.704541 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 400/817] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 3 04:55:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:05.705379 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b8fe8-c14e-4843-90c3-c206cd906a55 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:05.764338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f00e4e7-ea22-4f10-9937-5a5ac15d9545 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:05.806348 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c5c7d-eea8-461c-9f58-e516b2979b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:05.807029 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c5c7d-eea8-461c-9f58-e516b2979b74 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:05.807312 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c5c7d-eea8-461c-9f58-e516b2979b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:05.807584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c5c7d-eea8-461c-9f58-e516b2979b74 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:05.808214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c5c7d-eea8-461c-9f58-e516b2979b74 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:05.809715 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c5c7d-eea8-461c-9f58-e516b2979b74 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:05.819347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-a34c5c7d-eea8-461c-9f58-e516b2979b74 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 04:55:05.819347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:05.819347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 04:55:05.819347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 04:55:05.819347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 04:55:05.819347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 04:55:05.819347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 04:55:05.819347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:05.819347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 04:55:05.819347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:05.819347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:55:05.819347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:05.819347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:05.819347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:05.819347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:05.819347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:05.819347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:05.820320 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:05.820320 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:05.820320 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:05.820320 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:05.820320 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:05.820320 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:05.820320 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 04:55:05.820320 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 04:55:05.820320 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 04:55:05.820320 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 04:55:05.820320 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:05.820320 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:05.820320 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:05.820320 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:05.820320 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:05.820320 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:05.821176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:05.821176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:05.821176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:05.821176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:05.821176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:05.821176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:05.821176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:05.821176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:05.821176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:05.821176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:05.821176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 04:55:05.821176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:05.821176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 04:55:05.821176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:05.821176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 418/818] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 3 04:55:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:05.822580 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c5c7d-eea8-461c-9f58-e516b2979b74 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:05.832374 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b8fe8-c14e-4843-90c3-c206cd906a55 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:05.840638 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:05.869918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca0eaf-fd76-42b4-ab72-4fbe0645f94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:05.870731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca0eaf-fd76-42b4-ab72-4fbe0645f94d None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:05.871050 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca0eaf-fd76-42b4-ab72-4fbe0645f94d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:05.871331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca0eaf-fd76-42b4-ab72-4fbe0645f94d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:05.878607 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca0eaf-fd76-42b4-ab72-4fbe0645f94d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:05.880104 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca0eaf-fd76-42b4-ab72-4fbe0645f94d None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:05.889770 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:05.894757 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-b5ca0eaf-fd76-42b4-ab72-4fbe0645f94d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 04:55:05.894757 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:05.894757 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:05.894757 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:05.894757 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:05.894757 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:05.894757 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:05.894757 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:05.894757 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:05.894757 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:05.894757 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:05.894757 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:05.894757 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:55:05.894757 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:55:05.894757 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:55:05.894757 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:55:05.895945 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:05.895945 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:05.895945 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:55:05.895945 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:55:05.895945 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:55:05.895945 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:55:05.895945 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:05.895945 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:05.895945 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:05.895945 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:05.895945 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:05.895945 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:05.895945 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:05.895945 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:05.895945 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:05.895945 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:05.895945 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:05.895945 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:05.897051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:05.897051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:05.897051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:55:05.897051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:55:05.897051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:55:05.897051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:55:05.897051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:55:05.897051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:55:05.897051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 04:55:05.897051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:05.897051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 401/819] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Aug 3 04:55:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:05.897929 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca0eaf-fd76-42b4-ab72-4fbe0645f94d None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:05.940070 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c5c7d-eea8-461c-9f58-e516b2979b74 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:05.976520 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f57ec-ec68-4cf1-a24f-069d3c8968bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:05.977171 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f57ec-ec68-4cf1-a24f-069d3c8968bb None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:05.977449 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f57ec-ec68-4cf1-a24f-069d3c8968bb None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:05.977720 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f57ec-ec68-4cf1-a24f-069d3c8968bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:05.989965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f57ec-ec68-4cf1-a24f-069d3c8968bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:05.991666 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f57ec-ec68-4cf1-a24f-069d3c8968bb None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:06.006049 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca0eaf-fd76-42b4-ab72-4fbe0645f94d None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:06.006859 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-8f9f57ec-ec68-4cf1-a24f-069d3c8968bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:06.006859 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:06.006859 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:06.006859 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:06.006859 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:06.006859 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:06.006859 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:06.006859 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:06.006859 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:06.006859 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:06.006859 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:06.006859 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:06.006859 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:55:06.006859 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:55:06.006859 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:55:06.007741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:55:06.007741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:06.007741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:06.007741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:06.007741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:06.007741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:06.007741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:06.007741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:06.007741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:06.007741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:06.007741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:06.007741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:06.007741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:06.007741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:06.007741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:06.007741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:06.007741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:06.008680 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:55:06.008680 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:55:06.008680 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:06.008680 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:06.008680 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 419/820] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:06.008971 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f57ec-ec68-4cf1-a24f-069d3c8968bb None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:06.047051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d6b731-286a-44ff-b097-72d8aafc8396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:06.047663 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6b731-286a-44ff-b097-72d8aafc8396 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:06.047944 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6b731-286a-44ff-b097-72d8aafc8396 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:06.048217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6b731-286a-44ff-b097-72d8aafc8396 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:06.048885 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d6b731-286a-44ff-b097-72d8aafc8396 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:06.049188 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d6b731-286a-44ff-b097-72d8aafc8396 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:06.050549 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d6b731-286a-44ff-b097-72d8aafc8396 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:06.059962 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 402/821] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 3 04:55:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:06.059962 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d6b731-286a-44ff-b097-72d8aafc8396 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:06.090551 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:06.122409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f57ec-ec68-4cf1-a24f-069d3c8968bb None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:06.142049 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:06.159731 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574edea2-5692-4e10-b64a-784493754c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:06.160457 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574edea2-5692-4e10-b64a-784493754c0a None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:06.160746 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574edea2-5692-4e10-b64a-784493754c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:06.162084 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574edea2-5692-4e10-b64a-784493754c0a None admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:06.162930 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574edea2-5692-4e10-b64a-784493754c0a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:06.163247 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574edea2-5692-4e10-b64a-784493754c0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:06.164704 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574edea2-5692-4e10-b64a-784493754c0a None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:06.180561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d6b731-286a-44ff-b097-72d8aafc8396 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:06.184891 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 420/822] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Aug 3 04:55:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:06.186870 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574edea2-5692-4e10-b64a-784493754c0a None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:06.218628 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a0c60-ff22-4e35-9c5d-93282424faf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:06.219257 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a0c60-ff22-4e35-9c5d-93282424faf6 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:06.219540 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a0c60-ff22-4e35-9c5d-93282424faf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:06.219815 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a0c60-ff22-4e35-9c5d-93282424faf6 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:06.220523 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0a0c60-ff22-4e35-9c5d-93282424faf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:06.220805 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0a0c60-ff22-4e35-9c5d-93282424faf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:06.222170 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0a0c60-ff22-4e35-9c5d-93282424faf6 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:06.238267 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 403/823] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:06.238267 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a0c60-ff22-4e35-9c5d-93282424faf6 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:06.299031 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574edea2-5692-4e10-b64a-784493754c0a None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:06.338342 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603992e2-3fcc-4743-a350-ed18c68f34b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:06.338992 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603992e2-3fcc-4743-a350-ed18c68f34b3 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:06.339276 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603992e2-3fcc-4743-a350-ed18c68f34b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:06.339551 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603992e2-3fcc-4743-a350-ed18c68f34b3 None admin] PATH_INFO: `/v3/services` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:06.340226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603992e2-3fcc-4743-a350-ed18c68f34b3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:06.340528 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603992e2-3fcc-4743-a350-ed18c68f34b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:06.342510 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603992e2-3fcc-4743-a350-ed18c68f34b3 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:06.345384 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:06.347754 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 421/824] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 3 04:55:06 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:06.350177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603992e2-3fcc-4743-a350-ed18c68f34b3 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:06.360064 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a0c60-ff22-4e35-9c5d-93282424faf6 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:06.403125 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:06.407519 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ba8aa-3cc2-48a9-9572-04822c3050b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:06.408309 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ba8aa-3cc2-48a9-9572-04822c3050b0 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:06.408694 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ba8aa-3cc2-48a9-9572-04822c3050b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:06.409070 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ba8aa-3cc2-48a9-9572-04822c3050b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:06.487030 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603992e2-3fcc-4743-a350-ed18c68f34b3 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:06.492460 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ba8aa-3cc2-48a9-9572-04822c3050b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:06.493359 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ba8aa-3cc2-48a9-9572-04822c3050b0 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:06.540866 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782f470-ac50-4961-a5f2-fb4492c74202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:06.543625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782f470-ac50-4961-a5f2-fb4492c74202 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:06.543914 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782f470-ac50-4961-a5f2-fb4492c74202 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:06.544195 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782f470-ac50-4961-a5f2-fb4492c74202 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:06.555446 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782f470-ac50-4961-a5f2-fb4492c74202 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:06.557085 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782f470-ac50-4961-a5f2-fb4492c74202 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:06.570997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-4782f470-ac50-4961-a5f2-fb4492c74202 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 04:55:06.570997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:06.570997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:06.570997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:06.570997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:06.570997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:06.570997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:06.570997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:06.570997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:06.570997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:06.570997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:06.570997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:06.570997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:55:06.570997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:55:06.570997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:55:06.570997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:55:06.572206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:06.572206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:06.572206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:55:06.572206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:55:06.572206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:55:06.572206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:55:06.572206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:06.572206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:06.572206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:06.572206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:06.572206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:06.572206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:06.572206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:06.572206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:06.572206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:06.572206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:06.572206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:06.572206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:06.573314 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:06.573314 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:06.573314 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:55:06.573314 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:55:06.573314 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:55:06.573314 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:55:06.573314 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:55:06.573314 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:55:06.573314 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 04:55:06.573314 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:06.575563 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 422/825] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Aug 3 04:55:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:06.581654 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782f470-ac50-4961-a5f2-fb4492c74202 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:06.614881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:06.619723 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 404/826] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:55:06 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 04:55:06.625775 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ba8aa-3cc2-48a9-9572-04822c3050b0 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:06.688861 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:06.693108 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782f470-ac50-4961-a5f2-fb4492c74202 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:06.734138 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe24d45b-6f36-4da7-8790-e378d26a586f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:06.734759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24d45b-6f36-4da7-8790-e378d26a586f None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:06.735065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24d45b-6f36-4da7-8790-e378d26a586f None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:06.735338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24d45b-6f36-4da7-8790-e378d26a586f None admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:06.736128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe24d45b-6f36-4da7-8790-e378d26a586f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:06.736434 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe24d45b-6f36-4da7-8790-e378d26a586f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:06.737863 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe24d45b-6f36-4da7-8790-e378d26a586f None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:06.759661 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ba8aa-3cc2-48a9-9572-04822c3050b0 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:06.762046 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 423/827] 10.222.0.40 () {56 vars in 1149 bytes} [Wed Aug 3 04:55:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:06.763990 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe24d45b-6f36-4da7-8790-e378d26a586f None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:06.806582 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d54ffe-9cb8-4646-be08-b29e595765af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:06.807632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d54ffe-9cb8-4646-be08-b29e595765af None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:06.808555 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d54ffe-9cb8-4646-be08-b29e595765af None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:06.809024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d54ffe-9cb8-4646-be08-b29e595765af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:06.823771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d54ffe-9cb8-4646-be08-b29e595765af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:06.827006 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d54ffe-9cb8-4646-be08-b29e595765af None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:06.843870 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-94d54ffe-9cb8-4646-be08-b29e595765af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:06.843870 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:06.843870 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:06.843870 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:06.843870 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:06.843870 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:06.843870 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:06.843870 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:06.843870 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:06.843870 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:06.843870 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:06.843870 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:06.843870 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:55:06.843870 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:55:06.843870 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:55:06.844879 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:55:06.844879 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:06.844879 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:06.844879 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:06.844879 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:06.844879 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:06.844879 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:06.844879 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:06.844879 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:06.844879 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:06.844879 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:06.844879 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:06.844879 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:06.844879 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:06.844879 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:06.844879 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:06.844879 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:06.846012 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:55:06.846012 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:55:06.846012 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:06.846012 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:06.846250 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 405/828] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:06.847372 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71192bfe-7f89-43ec-a1df-62e04d265b16 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:06.847658 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71192bfe-7f89-43ec-a1df-62e04d265b16 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:06.847989 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71192bfe-7f89-43ec-a1df-62e04d265b16 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:06.849379 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 406/829] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:06.852580 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71192bfe-7f89-43ec-a1df-62e04d265b16 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:06.886048 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe24d45b-6f36-4da7-8790-e378d26a586f None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:06.899375 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:06.923782 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2a3fbc-2bca-4727-a061-1253c6c4a3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:06.924748 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a3fbc-2bca-4727-a061-1253c6c4a3a1 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:06.925206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a3fbc-2bca-4727-a061-1253c6c4a3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:06.925654 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a3fbc-2bca-4727-a061-1253c6c4a3a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:06.938242 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2a3fbc-2bca-4727-a061-1253c6c4a3a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:06.940257 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2a3fbc-2bca-4727-a061-1253c6c4a3a1 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:06.955765 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-0b2a3fbc-2bca-4727-a061-1253c6c4a3a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:06.955765 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:06.955765 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:06.955765 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:06.955765 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:06.955765 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:06.955765 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:06.955765 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:06.955765 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:06.955765 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:06.955765 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:06.955765 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:06.955765 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:55:06.955765 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:55:06.955765 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:55:06.956678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:55:06.956678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:06.956678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:06.956678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:06.956678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:06.956678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:06.956678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:06.956678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:06.956678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:06.956678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:06.956678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:06.956678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:06.956678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:06.956678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:06.956678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:06.956678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:06.956678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:06.957695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:55:06.957695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:55:06.957695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:06.957695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:06.957695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 424/830] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:06.958127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4a294-0e3b-42c5-a15c-79f0554f1cdb None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:06.958204 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4a294-0e3b-42c5-a15c-79f0554f1cdb None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:06.958787 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4a294-0e3b-42c5-a15c-79f0554f1cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:06.965358 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71192bfe-7f89-43ec-a1df-62e04d265b16 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:06.976778 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-a9e4a294-0e3b-42c5-a15c-79f0554f1cdb None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:06.977977 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-a9e4a294-0e3b-42c5-a15c-79f0554f1cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:06.980041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:07.003522 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba6068-d8e2-47fa-b93b-d5b20c36f5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:07.004445 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba6068-d8e2-47fa-b93b-d5b20c36f5c7 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:07.004750 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba6068-d8e2-47fa-b93b-d5b20c36f5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:07.004950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba6068-d8e2-47fa-b93b-d5b20c36f5c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:07.005644 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba6068-d8e2-47fa-b93b-d5b20c36f5c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:07.006000 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba6068-d8e2-47fa-b93b-d5b20c36f5c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:07.007418 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba6068-d8e2-47fa-b93b-d5b20c36f5c7 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:07.020280 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 407/831] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:07.025349 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba6068-d8e2-47fa-b93b-d5b20c36f5c7 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:07.049728 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 425/832] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:06 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:55:07.050746 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4a294-0e3b-42c5-a15c-79f0554f1cdb None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:07.151342 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba6068-d8e2-47fa-b93b-d5b20c36f5c7 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:07.153334 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:07.159596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4a294-0e3b-42c5-a15c-79f0554f1cdb None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:07.189405 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf3fe3-4fa7-49c9-ac65-89a3c1a2ebe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:07.190027 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf3fe3-4fa7-49c9-ac65-89a3c1a2ebe8 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:07.190312 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf3fe3-4fa7-49c9-ac65-89a3c1a2ebe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:07.190589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf3fe3-4fa7-49c9-ac65-89a3c1a2ebe8 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:07.191308 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf3fe3-4fa7-49c9-ac65-89a3c1a2ebe8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:07.191615 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf3fe3-4fa7-49c9-ac65-89a3c1a2ebe8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:07.192987 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf3fe3-4fa7-49c9-ac65-89a3c1a2ebe8 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:07.198347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec88b68-4a6b-4992-b207-30849f50e6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:07.199145 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec88b68-4a6b-4992-b207-30849f50e6d0 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:07.199428 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec88b68-4a6b-4992-b207-30849f50e6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:07.199703 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec88b68-4a6b-4992-b207-30849f50e6d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:07.209841 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 408/833] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:55:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:07.209841 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf3fe3-4fa7-49c9-ac65-89a3c1a2ebe8 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:07.211822 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec88b68-4a6b-4992-b207-30849f50e6d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:07.213295 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec88b68-4a6b-4992-b207-30849f50e6d0 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:07.215193 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:07.225759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-cec88b68-4a6b-4992-b207-30849f50e6d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:55:07.225759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:07.225759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:07.225759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:07.225759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:07.225759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:07.225759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:07.225759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:07.225759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:07.225759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:07.225759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:07.225759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:07.225759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:55:07.225759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:55:07.225759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:55:07.232909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:55:07.232909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:07.232909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:07.232909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:07.232909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:07.232909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:07.232909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:07.232909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:07.232909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:07.232909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:07.232909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:07.232909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:07.232909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:07.232909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:07.232909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:07.232909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:07.232909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:07.233825 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:55:07.233825 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:55:07.233825 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:55:07.233825 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:55:07.233825 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:55:07.233825 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:55:07.233825 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:55:07.233825 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:07.233825 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 426/834] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:55:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:07.233825 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec88b68-4a6b-4992-b207-30849f50e6d0 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:07.324012 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf3fe3-4fa7-49c9-ac65-89a3c1a2ebe8 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:07.331935 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec88b68-4a6b-4992-b207-30849f50e6d0 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:07.361919 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffec7c4-3d9c-4067-a3f8-f5ed35d2eacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:07.362557 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffec7c4-3d9c-4067-a3f8-f5ed35d2eacf None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:07.362863 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffec7c4-3d9c-4067-a3f8-f5ed35d2eacf None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:07.363155 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffec7c4-3d9c-4067-a3f8-f5ed35d2eacf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:07.367132 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a3373-9c10-49c6-a389-04cd27beba77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:07.367771 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a3373-9c10-49c6-a389-04cd27beba77 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:07.368099 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a3373-9c10-49c6-a389-04cd27beba77 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:07.368380 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a3373-9c10-49c6-a389-04cd27beba77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:07.373728 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffec7c4-3d9c-4067-a3f8-f5ed35d2eacf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:55:07.375444 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0a3373-9c10-49c6-a389-04cd27beba77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:07.376896 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0a3373-9c10-49c6-a389-04cd27beba77 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:07.379086 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-9ffec7c4-3d9c-4067-a3f8-f5ed35d2eacf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:55:07.379086 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:07.379086 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:07.379086 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:07.379086 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:07.379086 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:07.379086 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:07.379086 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:07.379086 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:07.379086 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:07.379086 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:07.379086 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:07.379086 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:55:07.379086 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:55:07.379086 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:55:07.379086 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:55:07.380081 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:55:07.380081 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:55:07.380081 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:55:07.380081 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:55:07.380081 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:07.380081 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:07.380081 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:07.380081 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:07.380081 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:07.380081 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:07.380081 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:07.380081 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:07.380081 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:07.380081 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:07.380081 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:07.380081 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:07.380951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:07.380951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:07.380951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:07.380951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:07.380951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:55:07.380951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:55:07.380951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:55:07.380951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:55:07.380951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:55:07.380951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:55:07.380951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:55:07.380951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:07.380951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 409/835] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:55:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:07.381659 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffec7c4-3d9c-4067-a3f8-f5ed35d2eacf None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:07.394522 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-7d0a3373-9c10-49c6-a389-04cd27beba77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:55:07.394522 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:07.394522 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:07.394522 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:07.394522 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:07.394522 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:07.394522 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:07.394522 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:07.394522 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:07.394522 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:07.394522 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:07.394522 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:07.394522 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:55:07.394522 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:55:07.394522 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:55:07.395541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:55:07.395541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:07.395541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:07.395541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:07.395541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:07.395541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:07.395541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:07.395541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:07.395541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:07.395541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:07.395541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:07.395541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:07.395541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:07.395541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:07.395541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:07.395541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:07.395541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:07.396441 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:55:07.396441 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:55:07.396441 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:55:07.396441 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:55:07.396441 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:55:07.396441 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:55:07.396441 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:55:07.396441 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:07.396441 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 427/836] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:55:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:07.396928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a3373-9c10-49c6-a389-04cd27beba77 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:07.413423 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:07.460031 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:07.498007 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffec7c4-3d9c-4067-a3f8-f5ed35d2eacf None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:07.505373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a3373-9c10-49c6-a389-04cd27beba77 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:07.540183 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa6157-63d5-45a2-bcd2-291923b3b8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:07.541072 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa6157-63d5-45a2-bcd2-291923b3b8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:07.541361 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa6157-63d5-45a2-bcd2-291923b3b8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:07.541641 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa6157-63d5-45a2-bcd2-291923b3b8f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:07.542392 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aa6157-63d5-45a2-bcd2-291923b3b8f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:07.542702 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aa6157-63d5-45a2-bcd2-291923b3b8f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:07.544203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aa6157-63d5-45a2-bcd2-291923b3b8f7 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:07.546445 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428d8fb-6bfa-40d8-a29d-20aaa28ec0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:07.547862 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428d8fb-6bfa-40d8-a29d-20aaa28ec0ee None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:07.548335 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428d8fb-6bfa-40d8-a29d-20aaa28ec0ee None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:07.548813 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428d8fb-6bfa-40d8-a29d-20aaa28ec0ee None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:07.549704 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f428d8fb-6bfa-40d8-a29d-20aaa28ec0ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:07.550192 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f428d8fb-6bfa-40d8-a29d-20aaa28ec0ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:07.552428 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f428d8fb-6bfa-40d8-a29d-20aaa28ec0ee None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:07.556963 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 410/837] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:55:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:07.557775 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa6157-63d5-45a2-bcd2-291923b3b8f7 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:07.566411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 428/838] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:55:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:07.567629 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428d8fb-6bfa-40d8-a29d-20aaa28ec0ee None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:07.654393 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:07.677581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa6157-63d5-45a2-bcd2-291923b3b8f7 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:07.686261 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428d8fb-6bfa-40d8-a29d-20aaa28ec0ee None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:07.707560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:07.715571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983f419-d04d-49ce-bff4-310aaefc34df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:07.716184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983f419-d04d-49ce-bff4-310aaefc34df None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:07.717024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983f419-d04d-49ce-bff4-310aaefc34df None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:07.728093 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983f419-d04d-49ce-bff4-310aaefc34df None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:07.728093 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983f419-d04d-49ce-bff4-310aaefc34df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:07.728093 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983f419-d04d-49ce-bff4-310aaefc34df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:07.728093 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983f419-d04d-49ce-bff4-310aaefc34df None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:07.728379 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d28e5a4-21b6-4b8f-a260-43377c54ee63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:07.728379 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28e5a4-21b6-4b8f-a260-43377c54ee63 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:07.728379 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28e5a4-21b6-4b8f-a260-43377c54ee63 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:07.728379 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28e5a4-21b6-4b8f-a260-43377c54ee63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:07.728850 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d28e5a4-21b6-4b8f-a260-43377c54ee63 None admin] RBAC: Adding query filter params (role.id=e39d4ecea4c64d5794e1079c9be3ad36, scope.project.id=cf3763a37383456184a9ca9351b4ce03, user.id=a3fa4a6c16274c18849869b55b33ac62) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:07.728850 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d28e5a4-21b6-4b8f-a260-43377c54ee63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:07.728850 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d28e5a4-21b6-4b8f-a260-43377c54ee63 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:07.742936 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 411/840] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:55:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:07.743148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 429/840] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:55:07 2022] GET /identity/v3/role_assignments?user.id=a3fa4a6c16274c18849869b55b33ac62&scope.project.id=cf3763a37383456184a9ca9351b4ce03&role.id=e39d4ecea4c64d5794e1079c9be3ad36 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:07.743243 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983f419-d04d-49ce-bff4-310aaefc34df None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:07.747537 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d28e5a4-21b6-4b8f-a260-43377c54ee63 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:07.847952 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983f419-d04d-49ce-bff4-310aaefc34df None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:07.872697 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d28e5a4-21b6-4b8f-a260-43377c54ee63 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:07.885413 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55391930-94fd-41d9-9cca-8e000ebf5097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:07.886165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55391930-94fd-41d9-9cca-8e000ebf5097 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:07.886522 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55391930-94fd-41d9-9cca-8e000ebf5097 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:07.886895 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55391930-94fd-41d9-9cca-8e000ebf5097 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:07.899887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55391930-94fd-41d9-9cca-8e000ebf5097 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:07.901478 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55391930-94fd-41d9-9cca-8e000ebf5097 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:07.903836 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:07.918402 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-55391930-94fd-41d9-9cca-8e000ebf5097 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:07.918402 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:07.918402 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:07.918402 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:07.918402 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:07.918402 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:07.918402 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:07.918402 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:07.918402 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:07.918402 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:07.918402 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:07.918402 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:07.918402 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:55:07.918402 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:55:07.918402 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:55:07.924924 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:55:07.924924 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:07.924924 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:07.924924 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:07.924924 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:07.924924 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:07.924924 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:07.924924 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:07.924924 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:07.924924 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:07.924924 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:07.924924 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:07.924924 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:07.924924 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:07.924924 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:07.924924 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:07.924924 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:07.926041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:55:07.926041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:55:07.926041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:07.926041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:07.926041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 412/841] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:07.926396 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1476a-9951-49e0-8779-e80e24725649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:07.926396 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1476a-9951-49e0-8779-e80e24725649 None admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:07.926396 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1476a-9951-49e0-8779-e80e24725649 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:07.926396 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1476a-9951-49e0-8779-e80e24725649 None admin] PATH_INFO: `/v3/projects/cf3763a37383456184a9ca9351b4ce03/users/de853a2dfffb4f05b6f47df2280ae4e1/roles/cae571342db14f5fac7849efe7f8636e` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:07.930613 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55391930-94fd-41d9-9cca-8e000ebf5097 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:07.953598 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1476a-9951-49e0-8779-e80e24725649 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf3763a37383456184a9ca9351b4ce03, user_id=de853a2dfffb4f05b6f47df2280ae4e1, role_id=cae571342db14f5fac7849efe7f8636e)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:07.955721 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1476a-9951-49e0-8779-e80e24725649 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:08.016647 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 430/842] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 3 04:55:07 2022] PUT /identity/v3/projects/cf3763a37383456184a9ca9351b4ce03/users/de853a2dfffb4f05b6f47df2280ae4e1/roles/cae571342db14f5fac7849efe7f8636e => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 04:55:08.054322 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55391930-94fd-41d9-9cca-8e000ebf5097 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:08.095358 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef792dc6-de39-41ac-8570-7e4eaa46863c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:08.096277 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef792dc6-de39-41ac-8570-7e4eaa46863c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:08.096732 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef792dc6-de39-41ac-8570-7e4eaa46863c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:08.097166 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef792dc6-de39-41ac-8570-7e4eaa46863c None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:08.098049 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef792dc6-de39-41ac-8570-7e4eaa46863c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:08.098525 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef792dc6-de39-41ac-8570-7e4eaa46863c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:08.100179 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef792dc6-de39-41ac-8570-7e4eaa46863c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:08.114691 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 413/843] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:55:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:08.123219 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1476a-9951-49e0-8779-e80e24725649 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:08.160082 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:08.232880 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1476a-9951-49e0-8779-e80e24725649 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:08.269714 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69cb83-08ed-431c-a2a6-91e64ecf1245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:08.270360 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69cb83-08ed-431c-a2a6-91e64ecf1245 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:08.270639 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69cb83-08ed-431c-a2a6-91e64ecf1245 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:08.270965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69cb83-08ed-431c-a2a6-91e64ecf1245 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:08.278978 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a69cb83-08ed-431c-a2a6-91e64ecf1245 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:08.280664 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a69cb83-08ed-431c-a2a6-91e64ecf1245 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:08.292909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-1a69cb83-08ed-431c-a2a6-91e64ecf1245 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 03 04:55:08.292909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:08.292909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:08.292909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:08.292909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:08.292909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:08.292909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:08.292909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:08.292909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:08.292909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:08.292909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:08.292909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:08.292909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:55:08.292909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:55:08.292909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:55:08.292909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:55:08.294120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:08.294120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:08.294120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:55:08.294120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:55:08.294120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:55:08.294120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:55:08.294120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:08.294120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:08.294120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:08.294120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:08.294120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:08.294120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:08.294120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:08.294120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:08.294120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:08.294120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:08.294120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:08.294120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:08.295550 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:08.295550 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:08.295550 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:55:08.295550 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:55:08.295550 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:55:08.295550 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:55:08.295550 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:55:08.295550 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:55:08.295550 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 03 04:55:08.295550 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:08.295550 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 431/844] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:55:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:08.300149 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef792dc6-de39-41ac-8570-7e4eaa46863c None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:08.362097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:08.410798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef792dc6-de39-41ac-8570-7e4eaa46863c None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:08.448277 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb6251-99a8-4ef9-a03e-fe193820dfde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:08.448897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb6251-99a8-4ef9-a03e-fe193820dfde None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:08.449165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb6251-99a8-4ef9-a03e-fe193820dfde None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:08.449434 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb6251-99a8-4ef9-a03e-fe193820dfde None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:08.450224 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb6251-99a8-4ef9-a03e-fe193820dfde None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:08.450580 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb6251-99a8-4ef9-a03e-fe193820dfde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:08.452165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb6251-99a8-4ef9-a03e-fe193820dfde None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:08.454053 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9447e-9f41-453b-a4c8-337e6b7c8dac None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:08.454331 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9447e-9f41-453b-a4c8-337e6b7c8dac None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:08.454600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9447e-9f41-453b-a4c8-337e6b7c8dac None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:08.456396 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 432/845] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:55:08.461381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680c963-42c0-4383-999f-9dc96f5648cc None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:08.461665 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680c963-42c0-4383-999f-9dc96f5648cc None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:08.461938 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680c963-42c0-4383-999f-9dc96f5648cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:08.476150 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 414/846] 10.222.0.40 () {56 vars in 1135 bytes} [Wed Aug 3 04:55:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:08.481380 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb6251-99a8-4ef9-a03e-fe193820dfde None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:08.486358 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-1680c963-42c0-4383-999f-9dc96f5648cc None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:08.487679 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-1680c963-42c0-4383-999f-9dc96f5648cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:08.584808 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 433/847] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:08 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 04:55:08.601520 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1680c963-42c0-4383-999f-9dc96f5648cc None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:08.610131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb6251-99a8-4ef9-a03e-fe193820dfde None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:08.629132 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:08.643667 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:08.662656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e002cf69-2f26-4554-bbc3-95785db8d58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:08.663682 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002cf69-2f26-4554-bbc3-95785db8d58e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:08.664433 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002cf69-2f26-4554-bbc3-95785db8d58e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:08.665015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002cf69-2f26-4554-bbc3-95785db8d58e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:08.742230 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e002cf69-2f26-4554-bbc3-95785db8d58e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:08.744074 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e002cf69-2f26-4554-bbc3-95785db8d58e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:08.744965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1680c963-42c0-4383-999f-9dc96f5648cc None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:08.788557 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ba093-da70-4d55-bfd8-835445ad1f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:08.789224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ba093-da70-4d55-bfd8-835445ad1f7a None admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:08.789509 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ba093-da70-4d55-bfd8-835445ad1f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:08.789789 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ba093-da70-4d55-bfd8-835445ad1f7a None admin] PATH_INFO: `/v3/services` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:08.790496 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96ba093-da70-4d55-bfd8-835445ad1f7a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:08.792299 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96ba093-da70-4d55-bfd8-835445ad1f7a None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:08.853804 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 415/848] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:55:08 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 03 04:55:08.860669 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e002cf69-2f26-4554-bbc3-95785db8d58e None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:08.873074 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 434/849] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Aug 3 04:55:08 2022] POST /identity/v3/services => generated 251 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 04:55:08.876706 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:08.979142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e002cf69-2f26-4554-bbc3-95785db8d58e None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:09.032335 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12f27e-b553-40bb-9b4a-d6d0e3ed26b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:09.033000 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12f27e-b553-40bb-9b4a-d6d0e3ed26b0 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:09.033280 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12f27e-b553-40bb-9b4a-d6d0e3ed26b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:09.033554 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12f27e-b553-40bb-9b4a-d6d0e3ed26b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:09.045698 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f12f27e-b553-40bb-9b4a-d6d0e3ed26b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:09.047251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f12f27e-b553-40bb-9b4a-d6d0e3ed26b0 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:09.062118 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-2f12f27e-b553-40bb-9b4a-d6d0e3ed26b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:09.062118 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:09.062118 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:09.062118 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:09.062118 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:09.062118 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:09.062118 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:09.062118 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:09.062118 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:09.062118 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:09.062118 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:09.062118 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:09.062118 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:55:09.062118 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:55:09.062118 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:55:09.063496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:55:09.063496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:09.063496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:09.063496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:09.063496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:09.063496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:09.063496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:09.063496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:09.063496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:09.063496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:09.063496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:09.063496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:09.063496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:09.063496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:09.063496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:09.063496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:09.063496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:09.064433 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:55:09.064433 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:55:09.064433 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:09.064433 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:09.066989 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 416/850] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:09.069318 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ba093-da70-4d55-bfd8-835445ad1f7a None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:09.161259 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:09.180093 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ba093-da70-4d55-bfd8-835445ad1f7a None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:09.220217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50624e2-19eb-4d4d-b18c-3982f77c2317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:09.220844 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50624e2-19eb-4d4d-b18c-3982f77c2317 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:09.221123 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50624e2-19eb-4d4d-b18c-3982f77c2317 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:09.221396 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50624e2-19eb-4d4d-b18c-3982f77c2317 None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:09.222085 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50624e2-19eb-4d4d-b18c-3982f77c2317 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:09.222457 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50624e2-19eb-4d4d-b18c-3982f77c2317 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:09.223951 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50624e2-19eb-4d4d-b18c-3982f77c2317 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:09.236756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 435/851] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:55:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:09.242366 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12f27e-b553-40bb-9b4a-d6d0e3ed26b0 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:09.309155 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:09.355074 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12f27e-b553-40bb-9b4a-d6d0e3ed26b0 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:09.391465 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f8b35-8e67-4f26-a6e4-c82b1debac3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:09.392473 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f8b35-8e67-4f26-a6e4-c82b1debac3c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:09.393108 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f8b35-8e67-4f26-a6e4-c82b1debac3c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:09.393588 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f8b35-8e67-4f26-a6e4-c82b1debac3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:09.401121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f8b35-8e67-4f26-a6e4-c82b1debac3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:09.402797 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f8b35-8e67-4f26-a6e4-c82b1debac3c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:09.416341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-111f8b35-8e67-4f26-a6e4-c82b1debac3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:55:09.416341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:09.416341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:09.416341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:09.416341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:09.416341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:09.416341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:09.416341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:09.416341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:09.416341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:09.416341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:09.416341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:09.416341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:55:09.416341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:55:09.416341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:55:09.417260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:55:09.417260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:09.417260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:09.417260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:09.417260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:09.417260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:09.417260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:09.417260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:09.417260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:09.417260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:09.417260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:09.417260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:09.417260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:09.417260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:09.417260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:09.417260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:09.417260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:09.418246 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:55:09.418246 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:55:09.418246 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:55:09.418246 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:55:09.418246 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:55:09.418246 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:55:09.418246 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:55:09.418246 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:09.424424 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 417/852] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:55:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:09.426748 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50624e2-19eb-4d4d-b18c-3982f77c2317 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:09.530349 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50624e2-19eb-4d4d-b18c-3982f77c2317 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:09.564949 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3325cd-57c3-47ff-8625-395e9b4b0b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:09.565488 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3325cd-57c3-47ff-8625-395e9b4b0b9d None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:09.565822 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3325cd-57c3-47ff-8625-395e9b4b0b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:09.566100 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3325cd-57c3-47ff-8625-395e9b4b0b9d None admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:09.566948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3325cd-57c3-47ff-8625-395e9b4b0b9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:09.567257 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3325cd-57c3-47ff-8625-395e9b4b0b9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:09.568680 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3325cd-57c3-47ff-8625-395e9b4b0b9d None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:09.581339 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 436/853] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:55:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:09.587370 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f8b35-8e67-4f26-a6e4-c82b1debac3c None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:09.709241 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f8b35-8e67-4f26-a6e4-c82b1debac3c None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:09.730293 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:09.744974 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d558e3d9-356d-41ff-8f96-099d3ba9708d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:09.746790 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558e3d9-356d-41ff-8f96-099d3ba9708d None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:09.746790 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558e3d9-356d-41ff-8f96-099d3ba9708d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:09.746790 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558e3d9-356d-41ff-8f96-099d3ba9708d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:09.747028 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d558e3d9-356d-41ff-8f96-099d3ba9708d None admin] RBAC: Adding query filter params (role.id=e39d4ecea4c64d5794e1079c9be3ad36, scope.project.id=cf3763a37383456184a9ca9351b4ce03, user.id=e1014bcb4d36441aa46042941c9a63d0) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:09.747217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d558e3d9-356d-41ff-8f96-099d3ba9708d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:09.749312 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d558e3d9-356d-41ff-8f96-099d3ba9708d None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:09.755820 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 418/854] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:55:09 2022] GET /identity/v3/role_assignments?user.id=e1014bcb4d36441aa46042941c9a63d0&scope.project.id=cf3763a37383456184a9ca9351b4ce03&role.id=e39d4ecea4c64d5794e1079c9be3ad36 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:09.975120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095c6af-8e16-42d8-aa71-bbe9947cce77 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:09.975619 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095c6af-8e16-42d8-aa71-bbe9947cce77 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:09.975619 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095c6af-8e16-42d8-aa71-bbe9947cce77 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:09.977202 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 437/855] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:09.983013 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9fcf7-05dd-46bf-9582-afd1c90cc1ee None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:09.983306 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9fcf7-05dd-46bf-9582-afd1c90cc1ee None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:09.983591 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9fcf7-05dd-46bf-9582-afd1c90cc1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:10.002611 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-e9e9fcf7-05dd-46bf-9582-afd1c90cc1ee None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:10.003844 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e9e9fcf7-05dd-46bf-9582-afd1c90cc1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:10.073794 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 419/856] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:09 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:55:10.095986 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2095c6af-8e16-42d8-aa71-bbe9947cce77 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:10.100043 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:10.212786 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2095c6af-8e16-42d8-aa71-bbe9947cce77 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:10.228648 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5315d-bcec-4d57-88f4-ca115bb64c50 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:10.228947 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5315d-bcec-4d57-88f4-ca115bb64c50 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:10.229149 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5315d-bcec-4d57-88f4-ca115bb64c50 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:10.230778 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 420/857] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:55:10.235994 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d507f7-9185-4982-b4ba-6b480430b6e5 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:10.236287 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d507f7-9185-4982-b4ba-6b480430b6e5 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:10.236561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d507f7-9185-4982-b4ba-6b480430b6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:10.253153 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c892898-ffe6-48b5-b100-a9d62563079c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:10.253863 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c892898-ffe6-48b5-b100-a9d62563079c None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:10.254151 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c892898-ffe6-48b5-b100-a9d62563079c None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:10.254427 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c892898-ffe6-48b5-b100-a9d62563079c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:10.255093 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c892898-ffe6-48b5-b100-a9d62563079c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:10.256209 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-93d507f7-9185-4982-b4ba-6b480430b6e5 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:10.256639 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c892898-ffe6-48b5-b100-a9d62563079c None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:10.257591 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-93d507f7-9185-4982-b4ba-6b480430b6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:10.267221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-2c892898-ffe6-48b5-b100-a9d62563079c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 04:55:10.267221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:10.267221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 04:55:10.267221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 04:55:10.267221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 04:55:10.267221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 04:55:10.267221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 04:55:10.267221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:10.267221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 04:55:10.267221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:10.267221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:55:10.267221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:10.267221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:10.267221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:10.267221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:10.267221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:10.268210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:10.268210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:10.268210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:10.268210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:10.268210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:10.268210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:10.268210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:10.268210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 04:55:10.268210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 04:55:10.268210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 04:55:10.268210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 04:55:10.268210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:10.268210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:10.268210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:10.268210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:10.268210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:10.269052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:10.269052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:10.269052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:10.269052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:10.269052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:10.269052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:10.269052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:10.269052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:10.269052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:10.269052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:10.269052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:10.269052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 04:55:10.269052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:10.269052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 04:55:10.269052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:10.270995 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 438/858] 10.222.0.40 () {56 vars in 1132 bytes} [Wed Aug 3 04:55:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:10.274860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c892898-ffe6-48b5-b100-a9d62563079c None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:10.338811 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 421/859] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:10 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 04:55:10.354707 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:10.362174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d507f7-9185-4982-b4ba-6b480430b6e5 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:10.401978 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c892898-ffe6-48b5-b100-a9d62563079c None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:10.420405 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:10.440597 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeab919-7464-4701-ac90-ce5bf2c0c792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:10.441529 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeab919-7464-4701-ac90-ce5bf2c0c792 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:10.441990 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeab919-7464-4701-ac90-ce5bf2c0c792 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:10.442440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeab919-7464-4701-ac90-ce5bf2c0c792 None admin] PATH_INFO: `/v3/services` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:10.443318 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeab919-7464-4701-ac90-ce5bf2c0c792 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:10.443809 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeab919-7464-4701-ac90-ce5bf2c0c792 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:10.445181 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeab919-7464-4701-ac90-ce5bf2c0c792 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:10.454876 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 439/860] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Aug 3 04:55:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:10.456764 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeab919-7464-4701-ac90-ce5bf2c0c792 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:10.491829 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d507f7-9185-4982-b4ba-6b480430b6e5 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:10.539297 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d661e24d-41d4-435d-a8bf-f52e2f75393c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:10.540324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661e24d-41d4-435d-a8bf-f52e2f75393c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:10.540828 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661e24d-41d4-435d-a8bf-f52e2f75393c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:10.541331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661e24d-41d4-435d-a8bf-f52e2f75393c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:10.552741 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d661e24d-41d4-435d-a8bf-f52e2f75393c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:55:10.558017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-d661e24d-41d4-435d-a8bf-f52e2f75393c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:55:10.558017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:10.558017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:10.558017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:10.558017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:10.558017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:10.558017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:10.558017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:10.558017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:10.558017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:10.558017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:10.558017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:10.558017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:55:10.558017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:55:10.558017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:55:10.558017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:55:10.558975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:55:10.558975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:55:10.558975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:55:10.558975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:55:10.558975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:10.558975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:10.558975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:10.558975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:10.558975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:10.558975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:10.558975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:10.558975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:10.558975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:10.558975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:10.558975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:10.558975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:10.559824 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:10.559824 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:10.559824 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:10.559824 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:10.559824 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:55:10.559824 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:55:10.559824 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:55:10.559824 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:55:10.559824 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:55:10.559824 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:55:10.559824 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:55:10.559824 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:10.567723 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 422/861] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 3 04:55:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:55:10.568848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d661e24d-41d4-435d-a8bf-f52e2f75393c None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:10.581548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeab919-7464-4701-ac90-ce5bf2c0c792 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:10.623740 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:10.630471 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0e9816-3f11-4b26-8fbb-ce06651a5859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:10.631417 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e9816-3f11-4b26-8fbb-ce06651a5859 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:10.640509 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e9816-3f11-4b26-8fbb-ce06651a5859 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:10.640509 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e9816-3f11-4b26-8fbb-ce06651a5859 None admin] PATH_INFO: `/v3/services` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:10.640509 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0e9816-3f11-4b26-8fbb-ce06651a5859 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:10.640509 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0e9816-3f11-4b26-8fbb-ce06651a5859 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:10.640509 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0e9816-3f11-4b26-8fbb-ce06651a5859 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:10.646888 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 440/862] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Aug 3 04:55:10 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:10.698696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d661e24d-41d4-435d-a8bf-f52e2f75393c None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:10.701524 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:10.739521 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb95a179-68a4-4a25-9a46-f49b818e90ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:10.740161 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95a179-68a4-4a25-9a46-f49b818e90ac None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:10.740441 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95a179-68a4-4a25-9a46-f49b818e90ac None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:10.740712 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95a179-68a4-4a25-9a46-f49b818e90ac None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:10.741521 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb95a179-68a4-4a25-9a46-f49b818e90ac None admin] RBAC: Adding query filter params (name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:10.741827 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb95a179-68a4-4a25-9a46-f49b818e90ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:10.743382 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb95a179-68a4-4a25-9a46-f49b818e90ac None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:10.754492 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 423/863] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 3 04:55:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:10.760774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0e9816-3f11-4b26-8fbb-ce06651a5859 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:10.876589 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0e9816-3f11-4b26-8fbb-ce06651a5859 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:10.914197 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c832c475-e352-4f40-859a-e5fed8a3885c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:10.915363 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c832c475-e352-4f40-859a-e5fed8a3885c None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:10.915856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c832c475-e352-4f40-859a-e5fed8a3885c None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:10.916395 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c832c475-e352-4f40-859a-e5fed8a3885c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:10.930398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c832c475-e352-4f40-859a-e5fed8a3885c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:10.932298 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c832c475-e352-4f40-859a-e5fed8a3885c None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:10.949230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-c832c475-e352-4f40-859a-e5fed8a3885c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:10.949230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:10.949230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:10.949230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:10.949230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:10.949230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:10.949230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:10.949230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:10.949230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:10.949230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:10.949230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:10.949230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:10.949230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:55:10.949230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:55:10.949230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:55:10.950252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:55:10.950252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:10.950252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:10.950252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:10.950252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:10.950252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:10.950252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:10.950252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:10.950252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:10.950252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:10.950252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:10.950252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:10.950252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:10.950252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:10.950252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:10.950252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:10.950252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:10.951232 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:55:10.951232 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:55:10.951232 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:10.951232 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:10.953502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 441/864] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:10.958860 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb95a179-68a4-4a25-9a46-f49b818e90ac None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:11.066635 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb95a179-68a4-4a25-9a46-f49b818e90ac None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:11.102285 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740d141f-8349-4524-b76e-90db7575f1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:11.102926 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d141f-8349-4524-b76e-90db7575f1cb None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:11.103206 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d141f-8349-4524-b76e-90db7575f1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:11.103484 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d141f-8349-4524-b76e-90db7575f1cb None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:11.104178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740d141f-8349-4524-b76e-90db7575f1cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:11.104483 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740d141f-8349-4524-b76e-90db7575f1cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:11.105969 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740d141f-8349-4524-b76e-90db7575f1cb None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:11.119333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 424/865] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:55:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:11.124983 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c832c475-e352-4f40-859a-e5fed8a3885c None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:11.178890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d9c60-ae1a-4f57-bdad-05c39ce27eb3 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:11.179204 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d9c60-ae1a-4f57-bdad-05c39ce27eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:11.179464 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d9c60-ae1a-4f57-bdad-05c39ce27eb3 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:11.181237 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 425/866] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:11.186952 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:11.187912 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640b366-2ddd-42a1-a618-4d11e200b1da None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:11.188192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640b366-2ddd-42a1-a618-4d11e200b1da None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:11.188466 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640b366-2ddd-42a1-a618-4d11e200b1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:11.206601 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-f640b366-2ddd-42a1-a618-4d11e200b1da None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:11.207683 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f640b366-2ddd-42a1-a618-4d11e200b1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:11.239233 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:11.257489 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c832c475-e352-4f40-859a-e5fed8a3885c None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:11.289426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 426/867] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:11 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:55:11.301760 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c573c8-5f9f-47c7-b3e4-95b102443572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:11.302410 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c573c8-5f9f-47c7-b3e4-95b102443572 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:11.302693 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c573c8-5f9f-47c7-b3e4-95b102443572 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:11.303015 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c573c8-5f9f-47c7-b3e4-95b102443572 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:11.309814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f640b366-2ddd-42a1-a618-4d11e200b1da None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:11.311188 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c573c8-5f9f-47c7-b3e4-95b102443572 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:11.312833 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c573c8-5f9f-47c7-b3e4-95b102443572 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:11.335972 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-50c573c8-5f9f-47c7-b3e4-95b102443572 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 04:55:11.335972 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:11.335972 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:11.335972 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:11.335972 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:11.335972 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:11.335972 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:11.335972 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:11.335972 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:11.335972 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:11.335972 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:11.335972 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:11.335972 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:55:11.335972 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:55:11.335972 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:55:11.335972 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:55:11.336990 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:11.336990 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:11.336990 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:55:11.336990 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:55:11.336990 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:55:11.336990 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:55:11.336990 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:11.336990 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:11.336990 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:11.336990 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:11.336990 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:11.336990 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:11.336990 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:11.336990 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:11.336990 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:11.336990 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:11.336990 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:11.336990 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:11.338338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:11.338338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:11.338338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:55:11.338338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:55:11.338338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:55:11.338338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:55:11.338338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:55:11.338338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:55:11.338338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 04:55:11.338338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:11.338338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 442/868] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Aug 3 04:55:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:11.338338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c573c8-5f9f-47c7-b3e4-95b102443572 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:11.431906 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f640b366-2ddd-42a1-a618-4d11e200b1da None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:11.470588 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c573c8-5f9f-47c7-b3e4-95b102443572 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:11.478439 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb141f6-0af3-4829-ab0e-f9b25b478ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:11.479120 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb141f6-0af3-4829-ab0e-f9b25b478ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:11.479405 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb141f6-0af3-4829-ab0e-f9b25b478ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:11.480245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb141f6-0af3-4829-ab0e-f9b25b478ef7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:11.480900 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb141f6-0af3-4829-ab0e-f9b25b478ef7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:11.482341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb141f6-0af3-4829-ab0e-f9b25b478ef7 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:11.483235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:11.495574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-1cb141f6-0af3-4829-ab0e-f9b25b478ef7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 04:55:11.495574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:11.495574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 04:55:11.495574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 04:55:11.495574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 04:55:11.495574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 04:55:11.495574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 04:55:11.495574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:11.495574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 04:55:11.495574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:11.495574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:55:11.495574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:11.495574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:11.495574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:11.495574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:11.495574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:11.495574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:11.497052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:11.497052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:11.497052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:11.497052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:11.497052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:11.497052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:11.497052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 04:55:11.497052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 04:55:11.497052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 04:55:11.497052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 04:55:11.497052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:11.497052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:11.497052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:11.497052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:11.497052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:11.497052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:11.497935 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:11.497935 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:11.497935 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:11.497935 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:11.497935 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:11.497935 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:11.497935 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:11.497935 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:11.497935 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:11.497935 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:11.497935 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 04:55:11.497935 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:11.497935 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 04:55:11.497935 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:11.500591 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 427/869] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 3 04:55:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:55:11.506643 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb141f6-0af3-4829-ab0e-f9b25b478ef7 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:11.518557 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31770c4-9553-4615-8f61-7f4506cb7dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:11.520151 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31770c4-9553-4615-8f61-7f4506cb7dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:11.520646 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31770c4-9553-4615-8f61-7f4506cb7dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:11.521153 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31770c4-9553-4615-8f61-7f4506cb7dc5 None admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:11.522918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31770c4-9553-4615-8f61-7f4506cb7dc5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:11.523849 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31770c4-9553-4615-8f61-7f4506cb7dc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:11.525838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31770c4-9553-4615-8f61-7f4506cb7dc5 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:11.535777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:11.552032 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 443/870] 10.222.0.40 () {56 vars in 1149 bytes} [Wed Aug 3 04:55:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:11.559832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31770c4-9553-4615-8f61-7f4506cb7dc5 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:11.626174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb141f6-0af3-4829-ab0e-f9b25b478ef7 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:11.667267 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31770c4-9553-4615-8f61-7f4506cb7dc5 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:11.667488 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e877b-1331-4bd4-9e53-2b8618760c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:11.668370 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e877b-1331-4bd4-9e53-2b8618760c14 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:11.668941 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e877b-1331-4bd4-9e53-2b8618760c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:11.669512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e877b-1331-4bd4-9e53-2b8618760c14 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:11.671089 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e877b-1331-4bd4-9e53-2b8618760c14 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:11.671936 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e877b-1331-4bd4-9e53-2b8618760c14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:11.674566 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e877b-1331-4bd4-9e53-2b8618760c14 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:11.686918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 428/871] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 3 04:55:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:11.688152 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e877b-1331-4bd4-9e53-2b8618760c14 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:11.714711 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606a07b2-b695-40e9-b2fb-7d3af237c7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:11.715476 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a07b2-b695-40e9-b2fb-7d3af237c7e6 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:11.715933 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a07b2-b695-40e9-b2fb-7d3af237c7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:11.716910 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a07b2-b695-40e9-b2fb-7d3af237c7e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:11.753423 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:11.792546 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:11.793526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606a07b2-b695-40e9-b2fb-7d3af237c7e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:11.795149 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606a07b2-b695-40e9-b2fb-7d3af237c7e6 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:11.811627 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e877b-1331-4bd4-9e53-2b8618760c14 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:11.851016 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290828ec-42cd-4660-99b4-759956681c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:11.851751 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290828ec-42cd-4660-99b4-759956681c43 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:11.852041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290828ec-42cd-4660-99b4-759956681c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:11.852319 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290828ec-42cd-4660-99b4-759956681c43 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:11.853023 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290828ec-42cd-4660-99b4-759956681c43 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:11.853326 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290828ec-42cd-4660-99b4-759956681c43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:11.854747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290828ec-42cd-4660-99b4-759956681c43 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:11.863214 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 429/872] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 3 04:55:11 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:11.863214 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ece56-8d19-4493-9cc3-4a7af2a76eb4 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:11.863214 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ece56-8d19-4493-9cc3-4a7af2a76eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:11.863214 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ece56-8d19-4493-9cc3-4a7af2a76eb4 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:11.863807 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 430/873] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:11.870510 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bb3aa-5e7c-44f2-9498-3009637d0a6b None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:11.871010 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bb3aa-5e7c-44f2-9498-3009637d0a6b None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:11.871295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bb3aa-5e7c-44f2-9498-3009637d0a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:11.892978 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-908bb3aa-5e7c-44f2-9498-3009637d0a6b None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:11.894121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-908bb3aa-5e7c-44f2-9498-3009637d0a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:11.909410 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 444/874] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:55:11 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 04:55:11.910149 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606a07b2-b695-40e9-b2fb-7d3af237c7e6 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:11.972250 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 431/875] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:11 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:55:11.973240 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908bb3aa-5e7c-44f2-9498-3009637d0a6b None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:11.988159 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:12.031954 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606a07b2-b695-40e9-b2fb-7d3af237c7e6 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:12.067432 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:12.069536 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060d0b8-183e-4e96-a120-d2fbcf34262e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:12.070151 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060d0b8-183e-4e96-a120-d2fbcf34262e None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:12.070430 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060d0b8-183e-4e96-a120-d2fbcf34262e None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:12.070697 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060d0b8-183e-4e96-a120-d2fbcf34262e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:12.071767 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5060d0b8-183e-4e96-a120-d2fbcf34262e None admin] RBAC: Adding query filter params (interface=public, service_id=276d7aaf87a54e3cb7a0a5798c6ab424, region_id=RegionOne) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:12.071767 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5060d0b8-183e-4e96-a120-d2fbcf34262e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:12.073129 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5060d0b8-183e-4e96-a120-d2fbcf34262e None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:12.083751 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 445/876] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Aug 3 04:55:11 2022] GET /identity/v3/endpoints?service_id=276d7aaf87a54e3cb7a0a5798c6ab424&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:12.083751 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060d0b8-183e-4e96-a120-d2fbcf34262e None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:12.097249 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908bb3aa-5e7c-44f2-9498-3009637d0a6b None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:12.133676 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da07a9-91c1-4bd9-bf13-0d0a2e051adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:12.134404 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da07a9-91c1-4bd9-bf13-0d0a2e051adf None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:12.134715 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da07a9-91c1-4bd9-bf13-0d0a2e051adf None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:12.135024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da07a9-91c1-4bd9-bf13-0d0a2e051adf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:12.147065 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da07a9-91c1-4bd9-bf13-0d0a2e051adf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:12.148834 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da07a9-91c1-4bd9-bf13-0d0a2e051adf None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:12.164534 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-c8da07a9-91c1-4bd9-bf13-0d0a2e051adf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:12.164534 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:12.164534 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:12.164534 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:12.164534 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:12.164534 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:12.164534 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:12.164534 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:12.164534 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:12.164534 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:12.164534 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:12.164534 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:12.164534 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:55:12.164534 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:55:12.164534 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:55:12.165440 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:55:12.165440 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:12.165440 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:12.165440 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:12.165440 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:12.165440 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:12.165440 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:12.165440 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:12.165440 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:12.165440 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:12.165440 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:12.165440 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:12.165440 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:12.165440 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:12.165440 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:12.165440 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:12.165440 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:12.166339 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:55:12.166339 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:55:12.166339 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:12.166339 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:12.166339 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 432/877] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:12.166617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da07a9-91c1-4bd9-bf13-0d0a2e051adf None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:12.194525 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060d0b8-183e-4e96-a120-d2fbcf34262e None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:12.224834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:12.230702 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63307d7-04da-4783-ab79-dd84e1c11d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:12.231375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63307d7-04da-4783-ab79-dd84e1c11d67 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:12.234545 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63307d7-04da-4783-ab79-dd84e1c11d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:12.234545 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63307d7-04da-4783-ab79-dd84e1c11d67 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:12.234545 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63307d7-04da-4783-ab79-dd84e1c11d67 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:12.238039 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63307d7-04da-4783-ab79-dd84e1c11d67 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:12.256701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-f63307d7-04da-4783-ab79-dd84e1c11d67 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 04:55:12.256701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:12.256701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 04:55:12.256701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 04:55:12.256701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 04:55:12.256701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 04:55:12.256701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 04:55:12.256701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:12.256701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 04:55:12.256701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:12.256701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:55:12.256701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:12.256701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:12.256701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:12.256701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:12.256701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:12.256701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:12.257772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:12.257772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:12.257772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:12.257772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:12.257772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:12.257772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:12.257772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 04:55:12.257772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 04:55:12.257772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 04:55:12.257772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 04:55:12.257772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:12.257772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:12.257772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:12.257772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:12.257772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:12.257772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:12.258621 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:12.258621 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:12.258621 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:12.258621 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:12.258621 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:12.258621 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:12.258621 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:12.258621 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:12.258621 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:12.258621 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:12.258621 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 04:55:12.258621 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:12.258621 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 04:55:12.258621 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:12.258621 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 446/878] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Aug 3 04:55:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:12.259471 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63307d7-04da-4783-ab79-dd84e1c11d67 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:12.287293 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da07a9-91c1-4bd9-bf13-0d0a2e051adf None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:12.319801 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:12.326393 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802db757-a474-476d-84a0-2d8b78bc5e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:12.327050 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802db757-a474-476d-84a0-2d8b78bc5e55 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:12.327342 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802db757-a474-476d-84a0-2d8b78bc5e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:12.327622 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802db757-a474-476d-84a0-2d8b78bc5e55 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:12.328281 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802db757-a474-476d-84a0-2d8b78bc5e55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:12.329672 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802db757-a474-476d-84a0-2d8b78bc5e55 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:12.347251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 433/879] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Aug 3 04:55:12 2022] GET /identity/v3/services => generated 1011 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 04:55:12.347251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802db757-a474-476d-84a0-2d8b78bc5e55 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:12.398883 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63307d7-04da-4783-ab79-dd84e1c11d67 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:12.448085 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1a781-639e-4e48-be02-a5d06532dea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:12.453980 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1a781-639e-4e48-be02-a5d06532dea9 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:12.453980 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1a781-639e-4e48-be02-a5d06532dea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:12.453980 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1a781-639e-4e48-be02-a5d06532dea9 None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:12.453980 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1a781-639e-4e48-be02-a5d06532dea9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:12.453980 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1a781-639e-4e48-be02-a5d06532dea9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:12.453980 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1a781-639e-4e48-be02-a5d06532dea9 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:12.470276 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 447/880] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:55:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:12.476183 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1a781-639e-4e48-be02-a5d06532dea9 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:12.486405 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802db757-a474-476d-84a0-2d8b78bc5e55 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:12.536757 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7ef47-838d-401c-a6e3-f585c0363b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:12.537395 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7ef47-838d-401c-a6e3-f585c0363b1d None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:12.537686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7ef47-838d-401c-a6e3-f585c0363b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:12.537967 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7ef47-838d-401c-a6e3-f585c0363b1d None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:12.538663 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d7ef47-838d-401c-a6e3-f585c0363b1d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:12.539026 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d7ef47-838d-401c-a6e3-f585c0363b1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:12.540797 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d7ef47-838d-401c-a6e3-f585c0363b1d None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:12.546119 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 434/881] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Aug 3 04:55:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:12.553360 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7ef47-838d-401c-a6e3-f585c0363b1d None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:12.555197 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:12.612482 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1a781-639e-4e48-be02-a5d06532dea9 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:12.635201 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:12.650379 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8567b-dfd2-4f8f-b2a5-33a94d46031d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:12.651094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8567b-dfd2-4f8f-b2a5-33a94d46031d None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:12.658597 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8567b-dfd2-4f8f-b2a5-33a94d46031d None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:12.658597 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8567b-dfd2-4f8f-b2a5-33a94d46031d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:12.660677 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b8567b-dfd2-4f8f-b2a5-33a94d46031d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:12.664760 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b8567b-dfd2-4f8f-b2a5-33a94d46031d None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:12.677485 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-35b8567b-dfd2-4f8f-b2a5-33a94d46031d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:55:12.677485 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:12.677485 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:12.677485 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:12.677485 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:12.677485 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:12.677485 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:12.677485 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:12.677485 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:12.677485 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:12.677485 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:12.677485 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:12.677485 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:55:12.677485 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:55:12.677485 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:55:12.678661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:55:12.678661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:12.678661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:12.678661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:12.678661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:12.678661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:12.678661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:12.678661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:12.678661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:12.678661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:12.678661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:12.678661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:12.678661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:12.678661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:12.678661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:12.678661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:12.678661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:12.679791 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:55:12.679791 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:55:12.679791 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:55:12.679791 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:55:12.679791 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:55:12.679791 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:55:12.679791 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:55:12.679791 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:12.686266 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 448/882] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:55:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:55:12.687511 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8567b-dfd2-4f8f-b2a5-33a94d46031d None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:12.689316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7ef47-838d-401c-a6e3-f585c0363b1d None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:12.730613 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72717e6e-5807-4a46-86a4-045ac009e018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:12.731458 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72717e6e-5807-4a46-86a4-045ac009e018 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:12.732481 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72717e6e-5807-4a46-86a4-045ac009e018 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:12.732858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72717e6e-5807-4a46-86a4-045ac009e018 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:12.733644 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72717e6e-5807-4a46-86a4-045ac009e018 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:12.734053 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72717e6e-5807-4a46-86a4-045ac009e018 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:12.735891 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72717e6e-5807-4a46-86a4-045ac009e018 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:12.747069 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 435/883] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Aug 3 04:55:12 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:12.789097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8bc4a2-8a07-4e8d-8649-d52976170248 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:12.789388 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8bc4a2-8a07-4e8d-8649-d52976170248 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:12.789672 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8bc4a2-8a07-4e8d-8649-d52976170248 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:12.791377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 436/884] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:55:12.797628 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167bc739-4aec-4428-8e31-74cf861ebfe3 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:12.798278 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167bc739-4aec-4428-8e31-74cf861ebfe3 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:12.798278 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167bc739-4aec-4428-8e31-74cf861ebfe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:12.809416 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8567b-dfd2-4f8f-b2a5-33a94d46031d None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:12.817547 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-167bc739-4aec-4428-8e31-74cf861ebfe3 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:12.818779 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-167bc739-4aec-4428-8e31-74cf861ebfe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:12.847266 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:12.858112 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa220166-9477-4574-b0e6-f4393f1d21d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:12.859282 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa220166-9477-4574-b0e6-f4393f1d21d0 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:12.859568 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa220166-9477-4574-b0e6-f4393f1d21d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:12.860774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa220166-9477-4574-b0e6-f4393f1d21d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:12.861564 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa220166-9477-4574-b0e6-f4393f1d21d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:12.861883 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa220166-9477-4574-b0e6-f4393f1d21d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:12.863408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa220166-9477-4574-b0e6-f4393f1d21d0 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:12.879847 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 449/885] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:55:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:12.885982 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa220166-9477-4574-b0e6-f4393f1d21d0 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:12.907412 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 437/886] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:12 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 04:55:12.927456 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167bc739-4aec-4428-8e31-74cf861ebfe3 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:12.990271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:13.019494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa220166-9477-4574-b0e6-f4393f1d21d0 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:13.042853 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167bc739-4aec-4428-8e31-74cf861ebfe3 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:13.070701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d934714-0c33-4879-8c46-45f15aa0f066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:13.072121 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d934714-0c33-4879-8c46-45f15aa0f066 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:13.072903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d934714-0c33-4879-8c46-45f15aa0f066 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:13.073331 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d934714-0c33-4879-8c46-45f15aa0f066 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:13.074966 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d934714-0c33-4879-8c46-45f15aa0f066 None admin] RBAC: Adding query filter params (role.id=cae571342db14f5fac7849efe7f8636e, scope.project.id=cf3763a37383456184a9ca9351b4ce03, user.id=de853a2dfffb4f05b6f47df2280ae4e1) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:13.075600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d934714-0c33-4879-8c46-45f15aa0f066 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:13.077597 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d934714-0c33-4879-8c46-45f15aa0f066 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:13.082861 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6146c0-e8d3-43bd-b250-a35d92d9f42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:13.083607 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6146c0-e8d3-43bd-b250-a35d92d9f42f None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:13.083993 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6146c0-e8d3-43bd-b250-a35d92d9f42f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:13.084371 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6146c0-e8d3-43bd-b250-a35d92d9f42f None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:13.085176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6146c0-e8d3-43bd-b250-a35d92d9f42f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:13.085761 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 450/887] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:55:12 2022] GET /identity/v3/role_assignments?user.id=de853a2dfffb4f05b6f47df2280ae4e1&scope.project.id=cf3763a37383456184a9ca9351b4ce03&role.id=cae571342db14f5fac7849efe7f8636e => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:13.096138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6146c0-e8d3-43bd-b250-a35d92d9f42f None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:13.209678 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 438/888] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Aug 3 04:55:12 2022] POST /identity/v3/services => generated 269 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:55:14.465563 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0db3f3-ed37-4c8c-b5e5-80b2f483fc51 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:14.466138 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0db3f3-ed37-4c8c-b5e5-80b2f483fc51 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:14.466219 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0db3f3-ed37-4c8c-b5e5-80b2f483fc51 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:14.467848 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 451/889] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:14.472959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb848808-463e-4a86-be6e-5666892b0d26 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:14.473355 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb848808-463e-4a86-be6e-5666892b0d26 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:14.473514 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb848808-463e-4a86-be6e-5666892b0d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:14.492419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-cb848808-463e-4a86-be6e-5666892b0d26 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:14.493629 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-cb848808-463e-4a86-be6e-5666892b0d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:14.570571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 439/890] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:14 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:55:14.592721 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0db3f3-ed37-4c8c-b5e5-80b2f483fc51 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:14.635119 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:14.703330 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0db3f3-ed37-4c8c-b5e5-80b2f483fc51 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:14.740307 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d548c9-4f8a-4713-9134-6c4f875843d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:14.740871 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d548c9-4f8a-4713-9134-6c4f875843d9 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:14.741149 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d548c9-4f8a-4713-9134-6c4f875843d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:14.741422 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d548c9-4f8a-4713-9134-6c4f875843d9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:14.742046 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d548c9-4f8a-4713-9134-6c4f875843d9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:14.743745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d548c9-4f8a-4713-9134-6c4f875843d9 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:14.753636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-c0d548c9-4f8a-4713-9134-6c4f875843d9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 04:55:14.753636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:14.753636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 04:55:14.753636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 04:55:14.753636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 04:55:14.753636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 04:55:14.753636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 04:55:14.753636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:14.753636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 04:55:14.753636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:14.753636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:55:14.753636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:14.753636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:14.753636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:14.753636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:14.753636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:14.753636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:14.754672 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:14.754672 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:14.754672 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:14.754672 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:14.754672 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:14.754672 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:14.754672 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 04:55:14.754672 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 04:55:14.754672 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 04:55:14.754672 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 04:55:14.754672 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:14.754672 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:14.754672 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:14.754672 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:14.754672 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:14.754672 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:14.755551 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:14.755551 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:14.755551 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:14.755551 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:14.755551 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:14.755551 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:14.755551 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:14.755551 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:14.755551 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:14.755551 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:14.755551 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 04:55:14.755551 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:14.755551 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 04:55:14.755551 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:14.758823 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 452/891] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 3 04:55:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:14.761211 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb848808-463e-4a86-be6e-5666892b0d26 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:14.803955 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:14.866912 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb848808-463e-4a86-be6e-5666892b0d26 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:14.895249 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2ca7a-f7c6-41b9-9053-ea932e7dd3fc None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:14.895559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2ca7a-f7c6-41b9-9053-ea932e7dd3fc None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:14.895798 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2ca7a-f7c6-41b9-9053-ea932e7dd3fc None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:14.897755 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 453/892] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:55:14.902698 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5404f7-ee99-4bd1-b90d-5a11ed68f201 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:14.903015 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5404f7-ee99-4bd1-b90d-5a11ed68f201 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:14.903295 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5404f7-ee99-4bd1-b90d-5a11ed68f201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:14.903996 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e96b236-5969-4b75-8ed2-13c4b9f92b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:14.904623 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96b236-5969-4b75-8ed2-13c4b9f92b36 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:14.904908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96b236-5969-4b75-8ed2-13c4b9f92b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:14.905360 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96b236-5969-4b75-8ed2-13c4b9f92b36 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:14.906052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e96b236-5969-4b75-8ed2-13c4b9f92b36 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:14.906359 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e96b236-5969-4b75-8ed2-13c4b9f92b36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:14.907929 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e96b236-5969-4b75-8ed2-13c4b9f92b36 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:14.914889 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 440/893] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 3 04:55:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:14.921015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e96b236-5969-4b75-8ed2-13c4b9f92b36 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:14.926666 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-8b5404f7-ee99-4bd1-b90d-5a11ed68f201 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:14.928229 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-8b5404f7-ee99-4bd1-b90d-5a11ed68f201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:15.022874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 454/894] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:14 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 04:55:15.039527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5404f7-ee99-4bd1-b90d-5a11ed68f201 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:15.043528 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e96b236-5969-4b75-8ed2-13c4b9f92b36 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:15.045345 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:15.078274 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:15.080202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec96e462-c5b5-4fa2-af86-3f0b63d5b517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:15.081055 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96e462-c5b5-4fa2-af86-3f0b63d5b517 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:15.081537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96e462-c5b5-4fa2-af86-3f0b63d5b517 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:15.082068 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96e462-c5b5-4fa2-af86-3f0b63d5b517 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:15.083440 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec96e462-c5b5-4fa2-af86-3f0b63d5b517 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:15.084006 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec96e462-c5b5-4fa2-af86-3f0b63d5b517 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:15.085966 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec96e462-c5b5-4fa2-af86-3f0b63d5b517 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:15.099068 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 441/895] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Aug 3 04:55:14 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:15.099861 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec96e462-c5b5-4fa2-af86-3f0b63d5b517 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:15.172634 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5404f7-ee99-4bd1-b90d-5a11ed68f201 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:15.209971 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117900de-925a-4749-8413-0c193e47795a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:15.211003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117900de-925a-4749-8413-0c193e47795a None admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:15.211485 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117900de-925a-4749-8413-0c193e47795a None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:15.211972 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117900de-925a-4749-8413-0c193e47795a None admin] PATH_INFO: `/v3/services` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:15.212856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117900de-925a-4749-8413-0c193e47795a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:15.214533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117900de-925a-4749-8413-0c193e47795a None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:15.227821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec96e462-c5b5-4fa2-af86-3f0b63d5b517 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:15.265214 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b5fa6-88b9-4acb-8831-c195c36a7f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:15.265881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b5fa6-88b9-4acb-8831-c195c36a7f04 None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:15.266193 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b5fa6-88b9-4acb-8831-c195c36a7f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:15.266464 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b5fa6-88b9-4acb-8831-c195c36a7f04 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:15.267196 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b5fa6-88b9-4acb-8831-c195c36a7f04 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:15.268565 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b5fa6-88b9-4acb-8831-c195c36a7f04 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:15.299107 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 455/896] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Aug 3 04:55:15 2022] POST /identity/v3/services => generated 241 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 04:55:15.300424 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b270d69-c212-4b30-bf0e-55ae7c94a9d4 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:15.300716 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b270d69-c212-4b30-bf0e-55ae7c94a9d4 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:15.300993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b270d69-c212-4b30-bf0e-55ae7c94a9d4 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:15.302210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 456/897] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:15.303567 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e6a45-bbba-4db3-bbec-50cba229aad5 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:15.303862 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e6a45-bbba-4db3-bbec-50cba229aad5 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:15.304137 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e6a45-bbba-4db3-bbec-50cba229aad5 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:15.305580 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 457/898] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:15.307819 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8155451-3bc9-43e5-b370-925cbdf5a0b6 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:15.308109 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8155451-3bc9-43e5-b370-925cbdf5a0b6 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:15.308394 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8155451-3bc9-43e5-b370-925cbdf5a0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:15.329735 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-f8155451-3bc9-43e5-b370-925cbdf5a0b6 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:15.331019 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-f8155451-3bc9-43e5-b370-925cbdf5a0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:15.365482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 442/899] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 04:55:15 2022] POST /identity/v3/endpoints => generated 346 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:55:15.365482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da5001-e959-4e1c-9b9f-ea29333adff3 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:15.365482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da5001-e959-4e1c-9b9f-ea29333adff3 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:15.365482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da5001-e959-4e1c-9b9f-ea29333adff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:15.384530 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-d8da5001-e959-4e1c-9b9f-ea29333adff3 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:15.385956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d8da5001-e959-4e1c-9b9f-ea29333adff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:15.412883 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:15.436858 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 458/900] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 04:55:15.462435 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8155451-3bc9-43e5-b370-925cbdf5a0b6 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:15.490382 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 443/901] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:55:15.498749 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:15.514696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da5001-e959-4e1c-9b9f-ea29333adff3 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:15.601744 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8155451-3bc9-43e5-b370-925cbdf5a0b6 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:15.639287 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da5001-e959-4e1c-9b9f-ea29333adff3 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:15.643300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64daff-c96e-41d0-b4da-3e4751d53d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:15.643969 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64daff-c96e-41d0-b4da-3e4751d53d07 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:15.644252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64daff-c96e-41d0-b4da-3e4751d53d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:15.644531 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64daff-c96e-41d0-b4da-3e4751d53d07 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:15.656519 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db64daff-c96e-41d0-b4da-3e4751d53d07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:55:15.663428 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-db64daff-c96e-41d0-b4da-3e4751d53d07 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 04:55:15.663428 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:15.663428 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:15.663428 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:15.663428 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:15.663428 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:15.663428 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:15.663428 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:15.663428 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:15.663428 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:15.663428 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:15.663428 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:15.663428 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:55:15.663428 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:55:15.663428 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:55:15.663428 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:55:15.665989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:55:15.665989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:55:15.665989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:55:15.665989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:55:15.665989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:15.665989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:15.665989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:15.665989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:15.665989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:15.665989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:15.665989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:15.665989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:15.665989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:15.665989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:15.665989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:15.665989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:15.667374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:15.667374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:15.667374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:15.667374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:15.667374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:55:15.667374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:55:15.667374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:55:15.667374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:55:15.667374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:55:15.667374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:55:15.667374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 04:55:15.667374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:15.667374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 459/902] 10.222.0.40 () {56 vars in 1120 bytes} [Wed Aug 3 04:55:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:55:15.670606 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64daff-c96e-41d0-b4da-3e4751d53d07 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:15.692452 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc2339-1a8d-43f6-8c2f-fae8924d1001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:15.693390 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc2339-1a8d-43f6-8c2f-fae8924d1001 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:15.693931 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc2339-1a8d-43f6-8c2f-fae8924d1001 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:15.694482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc2339-1a8d-43f6-8c2f-fae8924d1001 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:15.695325 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc2339-1a8d-43f6-8c2f-fae8924d1001 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:15.696913 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc2339-1a8d-43f6-8c2f-fae8924d1001 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:15.707918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-64dc2339-1a8d-43f6-8c2f-fae8924d1001 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 04:55:15.707918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:15.707918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 04:55:15.707918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 04:55:15.707918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 04:55:15.707918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 04:55:15.707918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 04:55:15.707918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:15.707918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 04:55:15.707918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:15.707918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:55:15.707918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:15.707918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:15.707918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:15.707918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:15.707918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:15.709124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:15.709124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:15.709124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:15.709124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:15.709124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:15.709124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:15.709124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:15.709124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 04:55:15.709124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 04:55:15.709124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 04:55:15.709124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 04:55:15.709124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:15.709124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:15.709124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:15.709124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:15.709124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:15.710035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:15.710035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:15.710035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:15.710035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:15.710035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:15.710035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:15.710035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:15.710035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:15.710035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:15.710035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:15.710035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:15.710035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 04:55:15.710035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:15.710035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 04:55:15.710035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:15.712589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 444/903] 10.222.0.40 () {56 vars in 1132 bytes} [Wed Aug 3 04:55:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:55:15.718674 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc2339-1a8d-43f6-8c2f-fae8924d1001 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:15.739341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:15.779366 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:15.802517 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64daff-c96e-41d0-b4da-3e4751d53d07 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:15.841766 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537aeb8-87ac-49ab-bb09-9e38cbcd95ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:15.842391 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537aeb8-87ac-49ab-bb09-9e38cbcd95ab None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:15.842674 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537aeb8-87ac-49ab-bb09-9e38cbcd95ab None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:15.843162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537aeb8-87ac-49ab-bb09-9e38cbcd95ab None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:15.843714 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537aeb8-87ac-49ab-bb09-9e38cbcd95ab None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:15.844093 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537aeb8-87ac-49ab-bb09-9e38cbcd95ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:15.845530 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537aeb8-87ac-49ab-bb09-9e38cbcd95ab None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:15.846582 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc2339-1a8d-43f6-8c2f-fae8924d1001 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:15.859182 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 460/904] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Aug 3 04:55:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:15.860667 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537aeb8-87ac-49ab-bb09-9e38cbcd95ab None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:15.889111 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82339646-70a8-49d3-b55f-a3e4b5cf1aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:15.890037 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82339646-70a8-49d3-b55f-a3e4b5cf1aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:15.890473 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82339646-70a8-49d3-b55f-a3e4b5cf1aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:15.890927 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82339646-70a8-49d3-b55f-a3e4b5cf1aa4 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:15.891873 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82339646-70a8-49d3-b55f-a3e4b5cf1aa4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:15.892455 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82339646-70a8-49d3-b55f-a3e4b5cf1aa4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:15.893979 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82339646-70a8-49d3-b55f-a3e4b5cf1aa4 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:15.905152 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 445/905] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Aug 3 04:55:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:15.906280 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82339646-70a8-49d3-b55f-a3e4b5cf1aa4 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:15.972874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537aeb8-87ac-49ab-bb09-9e38cbcd95ab None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:16.002399 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:16.009339 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a447e-81c8-4f80-b5ca-f99c1c9396a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:16.009992 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a447e-81c8-4f80-b5ca-f99c1c9396a3 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:16.010398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a447e-81c8-4f80-b5ca-f99c1c9396a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:16.010683 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a447e-81c8-4f80-b5ca-f99c1c9396a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:16.020365 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:16.023057 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a447e-81c8-4f80-b5ca-f99c1c9396a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:16.024553 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a447e-81c8-4f80-b5ca-f99c1c9396a3 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:16.027906 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82339646-70a8-49d3-b55f-a3e4b5cf1aa4 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:16.039832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-431a447e-81c8-4f80-b5ca-f99c1c9396a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:16.039832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:16.039832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:16.039832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:16.039832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:16.039832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:16.039832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:16.039832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:16.039832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:16.039832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:16.039832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:16.039832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:16.039832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:55:16.039832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:55:16.039832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:55:16.041003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:55:16.041003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:16.041003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:16.041003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:16.041003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:16.041003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:16.041003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:16.041003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:16.041003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:16.041003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:16.041003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:16.041003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:16.041003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:16.041003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:16.041003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:16.041003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:16.041003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:16.042122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:55:16.042122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:55:16.042122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:16.042122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:16.042122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 461/906] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:16.052180 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a447e-81c8-4f80-b5ca-f99c1c9396a3 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:16.069515 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ee85b-6e3a-4d5e-b6f9-b195b5c1b16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:16.070075 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ee85b-6e3a-4d5e-b6f9-b195b5c1b16e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:16.070365 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ee85b-6e3a-4d5e-b6f9-b195b5c1b16e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:16.070645 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ee85b-6e3a-4d5e-b6f9-b195b5c1b16e None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:16.071397 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ee85b-6e3a-4d5e-b6f9-b195b5c1b16e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:16.071710 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ee85b-6e3a-4d5e-b6f9-b195b5c1b16e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:16.073231 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ee85b-6e3a-4d5e-b6f9-b195b5c1b16e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:16.084024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 446/907] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Aug 3 04:55:15 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:16.084934 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ee85b-6e3a-4d5e-b6f9-b195b5c1b16e None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:16.197468 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a447e-81c8-4f80-b5ca-f99c1c9396a3 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:16.201863 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ee85b-6e3a-4d5e-b6f9-b195b5c1b16e None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:16.241958 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281ff6f-2194-417b-b55a-c3f69df888c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:16.242978 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281ff6f-2194-417b-b55a-c3f69df888c8 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:16.243372 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a5ee9-f1f9-42ae-a694-5b3f3581da83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:16.243567 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281ff6f-2194-417b-b55a-c3f69df888c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:16.243977 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281ff6f-2194-417b-b55a-c3f69df888c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:16.244266 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a5ee9-f1f9-42ae-a694-5b3f3581da83 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:16.244648 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a5ee9-f1f9-42ae-a694-5b3f3581da83 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:16.245039 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a5ee9-f1f9-42ae-a694-5b3f3581da83 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:16.245287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1281ff6f-2194-417b-b55a-c3f69df888c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:16.245731 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1281ff6f-2194-417b-b55a-c3f69df888c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:16.246119 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a5ee9-f1f9-42ae-a694-5b3f3581da83 None admin] RBAC: Adding query filter params (interface=public, service_id=575c80736693410a99962ea426655465, region_id=RegionOne) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:16.246529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a5ee9-f1f9-42ae-a694-5b3f3581da83 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:16.248271 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1281ff6f-2194-417b-b55a-c3f69df888c8 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:16.248849 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a5ee9-f1f9-42ae-a694-5b3f3581da83 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:16.260630 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 447/908] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Aug 3 04:55:16 2022] GET /identity/v3/endpoints?service_id=575c80736693410a99962ea426655465&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:16.261599 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a5ee9-f1f9-42ae-a694-5b3f3581da83 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:16.267124 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 462/909] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:55:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:16.269044 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281ff6f-2194-417b-b55a-c3f69df888c8 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:16.279320 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:16.344262 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:16.388204 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a5ee9-f1f9-42ae-a694-5b3f3581da83 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:16.412870 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281ff6f-2194-417b-b55a-c3f69df888c8 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:16.426456 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e9b12-9be6-4c4b-bacd-464c35042a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:16.426930 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e9b12-9be6-4c4b-bacd-464c35042a9d None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:16.427217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e9b12-9be6-4c4b-bacd-464c35042a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:16.427495 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e9b12-9be6-4c4b-bacd-464c35042a9d None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:16.428144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401e9b12-9be6-4c4b-bacd-464c35042a9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:16.429513 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401e9b12-9be6-4c4b-bacd-464c35042a9d None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:16.442620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 448/910] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Aug 3 04:55:16 2022] GET /identity/v3/services => generated 1497 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 04:55:16.457163 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ed03c-e5aa-4bfd-846f-4b755f284b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:16.458186 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ed03c-e5aa-4bfd-846f-4b755f284b12 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:16.458672 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ed03c-e5aa-4bfd-846f-4b755f284b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:16.459199 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ed03c-e5aa-4bfd-846f-4b755f284b12 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:16.469040 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ed03c-e5aa-4bfd-846f-4b755f284b12 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:16.470905 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ed03c-e5aa-4bfd-846f-4b755f284b12 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:16.485214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-599ed03c-e5aa-4bfd-846f-4b755f284b12 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 04:55:16.485214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:16.485214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:16.485214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:16.485214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:16.485214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:16.485214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:16.485214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:16.485214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:16.485214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:16.485214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:16.485214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:16.485214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:55:16.485214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:55:16.485214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:55:16.485214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:55:16.486241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:16.486241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:16.486241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:55:16.486241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:55:16.486241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:55:16.486241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:55:16.486241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:16.486241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:16.486241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:16.486241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:16.486241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:16.486241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:16.486241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:16.486241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:16.486241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:16.486241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:16.486241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:16.486241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:16.487239 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:16.487239 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:16.487239 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:55:16.487239 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:55:16.487239 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:55:16.487239 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:55:16.487239 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:55:16.487239 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:55:16.487239 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 04:55:16.487239 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:16.490032 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 463/911] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Aug 3 04:55:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:16.496161 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e9b12-9be6-4c4b-bacd-464c35042a9d None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:16.587105 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:16.625298 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e9b12-9be6-4c4b-bacd-464c35042a9d None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:16.668839 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e544b5b-0af3-4902-b694-08be514dbc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:16.670244 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e544b5b-0af3-4902-b694-08be514dbc40 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:16.670895 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e544b5b-0af3-4902-b694-08be514dbc40 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:16.671355 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e544b5b-0af3-4902-b694-08be514dbc40 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:16.672324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e544b5b-0af3-4902-b694-08be514dbc40 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:16.672958 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e544b5b-0af3-4902-b694-08be514dbc40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:16.674622 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e544b5b-0af3-4902-b694-08be514dbc40 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:16.694509 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 449/912] 10.222.0.40 () {56 vars in 1149 bytes} [Wed Aug 3 04:55:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:16.699775 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ed03c-e5aa-4bfd-846f-4b755f284b12 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:16.803792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ed03c-e5aa-4bfd-846f-4b755f284b12 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:16.817253 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:16.843021 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cb2094-a035-4dab-bc2f-b2869350a134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:16.843699 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb2094-a035-4dab-bc2f-b2869350a134 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:16.843984 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb2094-a035-4dab-bc2f-b2869350a134 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:16.844260 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb2094-a035-4dab-bc2f-b2869350a134 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:16.917582 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cb2094-a035-4dab-bc2f-b2869350a134 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:16.919628 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cb2094-a035-4dab-bc2f-b2869350a134 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:17.027065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 464/913] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:55:16 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 04:55:17.033284 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e544b5b-0af3-4902-b694-08be514dbc40 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:17.139970 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e544b5b-0af3-4902-b694-08be514dbc40 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:17.172728 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:17.179545 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f2a46-10cc-4b09-af01-d828215c4ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:17.180445 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f2a46-10cc-4b09-af01-d828215c4ddf None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:17.180927 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f2a46-10cc-4b09-af01-d828215c4ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:17.181400 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f2a46-10cc-4b09-af01-d828215c4ddf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:17.192144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f2a46-10cc-4b09-af01-d828215c4ddf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:17.198057 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f2a46-10cc-4b09-af01-d828215c4ddf None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:17.209300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-af9f2a46-10cc-4b09-af01-d828215c4ddf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:17.209300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:17.209300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:17.209300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:17.209300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:17.209300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:17.209300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:17.209300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:17.209300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:17.209300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:17.209300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:17.209300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:17.209300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:55:17.209300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:55:17.209300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:55:17.210203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:55:17.210203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:17.210203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:17.210203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:17.210203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:17.210203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:17.210203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:17.210203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:17.210203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:17.210203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:17.210203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:17.210203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:17.210203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:17.210203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:17.210203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:17.210203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:17.210203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:17.211123 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:55:17.211123 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:55:17.211123 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:17.211123 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:17.212924 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 450/914] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:17.219278 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cb2094-a035-4dab-bc2f-b2869350a134 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:17.235767 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:17.325533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cb2094-a035-4dab-bc2f-b2869350a134 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:17.369954 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54cc3a9-89e2-40b2-9ff4-fdf77f2825f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:17.370581 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54cc3a9-89e2-40b2-9ff4-fdf77f2825f1 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:17.370880 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54cc3a9-89e2-40b2-9ff4-fdf77f2825f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:17.371165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54cc3a9-89e2-40b2-9ff4-fdf77f2825f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:17.371860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54cc3a9-89e2-40b2-9ff4-fdf77f2825f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:17.372196 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54cc3a9-89e2-40b2-9ff4-fdf77f2825f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:17.373575 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54cc3a9-89e2-40b2-9ff4-fdf77f2825f1 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:17.385747 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 465/915] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:55:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:17.391100 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f2a46-10cc-4b09-af01-d828215c4ddf None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:17.457074 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca53e8-8c37-4e30-ab5f-f196d6053835 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:17.457390 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca53e8-8c37-4e30-ab5f-f196d6053835 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:17.457634 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca53e8-8c37-4e30-ab5f-f196d6053835 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:17.459263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 466/916] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:17.464985 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d46ef-e6c0-449a-b046-dc4d0bc70258 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:17.465360 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d46ef-e6c0-449a-b046-dc4d0bc70258 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:17.465714 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d46ef-e6c0-449a-b046-dc4d0bc70258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:17.485944 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-892d46ef-e6c0-449a-b046-dc4d0bc70258 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:17.487305 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-892d46ef-e6c0-449a-b046-dc4d0bc70258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:17.500348 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f2a46-10cc-4b09-af01-d828215c4ddf None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:17.536289 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ef14a-a1a5-44a2-bac1-4e5b2fc96e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:17.536937 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ef14a-a1a5-44a2-bac1-4e5b2fc96e63 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:17.537218 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ef14a-a1a5-44a2-bac1-4e5b2fc96e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:17.537493 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ef14a-a1a5-44a2-bac1-4e5b2fc96e63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:17.544695 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ef14a-a1a5-44a2-bac1-4e5b2fc96e63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:17.546201 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ef14a-a1a5-44a2-bac1-4e5b2fc96e63 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:17.558190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-594ef14a-a1a5-44a2-bac1-4e5b2fc96e63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:55:17.558190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:17.558190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:17.558190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:17.558190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:17.558190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:17.558190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:17.558190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:17.558190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:17.558190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:17.558190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:17.558190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:17.558190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:55:17.558190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:55:17.558190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:55:17.559143 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:55:17.559143 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:17.559143 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:17.559143 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:17.559143 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:17.559143 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:17.559143 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:17.559143 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:17.559143 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:17.559143 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:17.559143 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:17.559143 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:17.559143 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:17.559143 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:17.559143 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:17.559143 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:17.559143 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:17.560061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:55:17.560061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:55:17.560061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:55:17.560061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:55:17.560061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:55:17.560061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:55:17.560061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:55:17.560061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:17.564536 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 451/917] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:55:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:17.565379 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ef14a-a1a5-44a2-bac1-4e5b2fc96e63 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:17.566999 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 467/918] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:55:17.568771 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95831b91-9b31-4d7f-adf4-09a1af7bb02f None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:17.569085 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95831b91-9b31-4d7f-adf4-09a1af7bb02f None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:17.569349 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95831b91-9b31-4d7f-adf4-09a1af7bb02f None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:17.574395 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 468/919] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:55:17.576361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826c370-830e-49f5-82a2-8dd182dcf8d3 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:17.576648 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826c370-830e-49f5-82a2-8dd182dcf8d3 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:17.576930 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826c370-830e-49f5-82a2-8dd182dcf8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:17.586928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:17.598286 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-9826c370-830e-49f5-82a2-8dd182dcf8d3 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:17.599526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-9826c370-830e-49f5-82a2-8dd182dcf8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:17.613305 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:17.673285 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 469/920] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 04:55:17.675736 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9826c370-830e-49f5-82a2-8dd182dcf8d3 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:17.684066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ef14a-a1a5-44a2-bac1-4e5b2fc96e63 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:17.722800 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52383b99-671a-4170-8528-6e3dd1d861d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:17.723528 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52383b99-671a-4170-8528-6e3dd1d861d4 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:17.723819 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52383b99-671a-4170-8528-6e3dd1d861d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:17.724101 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52383b99-671a-4170-8528-6e3dd1d861d4 None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:17.724861 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52383b99-671a-4170-8528-6e3dd1d861d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:17.725175 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52383b99-671a-4170-8528-6e3dd1d861d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:17.726621 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52383b99-671a-4170-8528-6e3dd1d861d4 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:17.739387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 452/921] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:55:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:17.740165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52383b99-671a-4170-8528-6e3dd1d861d4 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:17.796522 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9826c370-830e-49f5-82a2-8dd182dcf8d3 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:17.828270 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:17.834264 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2299ba-e224-4548-ba3e-b9c1311994a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:17.835127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2299ba-e224-4548-ba3e-b9c1311994a4 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:17.835315 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2299ba-e224-4548-ba3e-b9c1311994a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:17.835593 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2299ba-e224-4548-ba3e-b9c1311994a4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:17.836226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2299ba-e224-4548-ba3e-b9c1311994a4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:17.837973 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2299ba-e224-4548-ba3e-b9c1311994a4 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:17.847515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-0d2299ba-e224-4548-ba3e-b9c1311994a4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 04:55:17.847515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:17.847515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 04:55:17.847515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 04:55:17.847515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 04:55:17.847515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 04:55:17.847515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 04:55:17.847515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:17.847515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 04:55:17.847515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:17.847515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:55:17.847515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:17.847515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:17.847515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:17.847515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:17.847515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:17.847515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:17.848473 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:17.848473 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:17.848473 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:17.848473 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:17.848473 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:17.848473 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:17.848473 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 04:55:17.848473 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 04:55:17.848473 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 04:55:17.848473 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 04:55:17.848473 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:17.848473 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:17.848473 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:17.848473 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:17.848473 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:17.848473 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:17.854714 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:17.854714 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:17.854714 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:17.854714 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:17.854714 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:17.854714 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:17.854714 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:17.854714 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:17.854714 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:17.854714 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:17.854714 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 04:55:17.854714 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:17.854714 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 04:55:17.854714 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:17.854714 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 470/922] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Aug 3 04:55:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:17.855605 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52383b99-671a-4170-8528-6e3dd1d861d4 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:17.855684 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2299ba-e224-4548-ba3e-b9c1311994a4 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:17.855757 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:17.893138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab67f680-37d4-407f-9dec-cc446ff1da4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:17.894099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67f680-37d4-407f-9dec-cc446ff1da4b None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:17.895049 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67f680-37d4-407f-9dec-cc446ff1da4b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:17.895557 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67f680-37d4-407f-9dec-cc446ff1da4b None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:17.896443 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab67f680-37d4-407f-9dec-cc446ff1da4b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:17.899036 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab67f680-37d4-407f-9dec-cc446ff1da4b None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:17.902946 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: WARNING keystone.server.flask.common [None req-ab67f680-37d4-407f-9dec-cc446ff1da4b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 03 04:55:17.944584 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.sql.core [None req-ab67f680-37d4-407f-9dec-cc446ff1da4b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 03 04:55:17.944584 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 03 04:55:17.944584 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [parameters: {'id': '3d7f49d88a9044ddb46ac7d9015f501a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 03 04:55:17.944584 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80355) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 03 04:55:17.955504 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2299ba-e224-4548-ba3e-b9c1311994a4 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:17.962798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-ab67f680-37d4-407f-9dec-cc446ff1da4b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3d7f49d88a9044ddb46ac7d9015f501a.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3d7f49d88a9044ddb46ac7d9015f501a. Aug 03 04:55:17.962798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:17.962798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 03 04:55:17.962798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 03 04:55:17.962798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 03 04:55:17.962798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 03 04:55:17.962798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 03 04:55:17.962798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application result = self._query(query) Aug 03 04:55:17.962798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 03 04:55:17.962798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application conn.query(q) Aug 03 04:55:17.962798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 03 04:55:17.962798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 03 04:55:17.962798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 03 04:55:17.963731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application result.read() Aug 03 04:55:17.963731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 03 04:55:17.963731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 03 04:55:17.963731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 03 04:55:17.963731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 03 04:55:17.963731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 03 04:55:17.963731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 03 04:55:17.963731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 03 04:55:17.963731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 03 04:55:17.963731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 03 04:55:17.963731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:17.963731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 03 04:55:17.963731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:17.963731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:17.963731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 03 04:55:17.963731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 03 04:55:17.963731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 03 04:55:17.963731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 03 04:55:17.964686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 03 04:55:17.964686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application next(self.gen) Aug 03 04:55:17.964686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 03 04:55:17.964686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application yield resource Aug 03 04:55:17.964686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 03 04:55:17.964686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application next(self.gen) Aug 03 04:55:17.964686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 03 04:55:17.964686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self.session.rollback() Aug 03 04:55:17.964686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 03 04:55:17.964686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self.force_reraise() Aug 03 04:55:17.964686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 03 04:55:17.964686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise self.value Aug 03 04:55:17.964686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 03 04:55:17.964686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 03 04:55:17.964686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 03 04:55:17.964686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application session.commit() Aug 03 04:55:17.964686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 03 04:55:17.965987 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 03 04:55:17.965987 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 03 04:55:17.965987 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._prepare_impl() Aug 03 04:55:17.965987 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 03 04:55:17.965987 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self.session.flush() Aug 03 04:55:17.965987 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 03 04:55:17.965987 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._flush(objects) Aug 03 04:55:17.965987 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 03 04:55:17.965987 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 03 04:55:17.965987 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 03 04:55:17.965987 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application compat.raise_( Aug 03 04:55:17.965987 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 03 04:55:17.965987 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception Aug 03 04:55:17.965987 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 03 04:55:17.965987 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application flush_context.execute() Aug 03 04:55:17.965987 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 03 04:55:17.965987 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rec.execute(self) Aug 03 04:55:17.966890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 03 04:55:17.966890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 03 04:55:17.966890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 03 04:55:17.966890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 03 04:55:17.966890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 03 04:55:17.966890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 03 04:55:17.966890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 03 04:55:17.966890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 03 04:55:17.966890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 03 04:55:17.966890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 03 04:55:17.966890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 03 04:55:17.966890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 03 04:55:17.966890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 03 04:55:17.966890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 03 04:55:17.966890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 03 04:55:17.968400 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 03 04:55:17.968400 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 03 04:55:17.968400 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception Aug 03 04:55:17.968400 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 03 04:55:17.968400 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 03 04:55:17.968400 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 03 04:55:17.968400 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 03 04:55:17.968400 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 03 04:55:17.968400 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application result = self._query(query) Aug 03 04:55:17.968400 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 03 04:55:17.968400 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application conn.query(q) Aug 03 04:55:17.968400 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 03 04:55:17.968400 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 03 04:55:17.968400 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 03 04:55:17.968400 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application result.read() Aug 03 04:55:17.968400 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 03 04:55:17.968400 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 03 04:55:17.970702 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 03 04:55:17.970702 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 03 04:55:17.970702 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 03 04:55:17.970702 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 03 04:55:17.970702 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 03 04:55:17.970702 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 03 04:55:17.970702 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 03 04:55:17.970702 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 03 04:55:17.970702 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [parameters: {'id': '3d7f49d88a9044ddb46ac7d9015f501a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 03 04:55:17.970702 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 03 04:55:17.970702 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:17.970702 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:55:17.970702 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:17.970702 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:17.970702 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 03 04:55:17.973570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 03 04:55:17.973570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 03 04:55:17.973570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 03 04:55:17.973570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 03 04:55:17.973570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:17.973570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:55:17.973570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:17.973570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:17.973570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:17.973570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:17.973570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:17.973570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:17.973570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:17.973570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:17.973570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:17.973570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:17.973570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:17.975774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:17.975774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 03 04:55:17.975774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 03 04:55:17.975774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:17.975774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:17.975774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 03 04:55:17.975774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 03 04:55:17.975774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3d7f49d88a9044ddb46ac7d9015f501a. Aug 03 04:55:17.975774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:17.975774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 453/923] 10.222.0.40 () {60 vars in 1140 bytes} [Wed Aug 3 04:55:17 2022] POST /identity/v3/projects => generated 262 bytes in 224 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:17.975774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab67f680-37d4-407f-9dec-cc446ff1da4b None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:17.996303 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b841a5a9-4377-4ea6-b584-a2ce798d604b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:17.997782 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b841a5a9-4377-4ea6-b584-a2ce798d604b None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:17.998264 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b841a5a9-4377-4ea6-b584-a2ce798d604b None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:17.998537 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b841a5a9-4377-4ea6-b584-a2ce798d604b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:17.999394 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b841a5a9-4377-4ea6-b584-a2ce798d604b None admin] RBAC: Adding query filter params (role.id=729f6ba7aded468788a72fefb5763fc4, scope.project.id=cf3763a37383456184a9ca9351b4ce03, user.id=de853a2dfffb4f05b6f47df2280ae4e1) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:17.999709 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b841a5a9-4377-4ea6-b584-a2ce798d604b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:18.001184 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b841a5a9-4377-4ea6-b584-a2ce798d604b None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:18.006805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 471/924] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:55:17 2022] GET /identity/v3/role_assignments?user.id=de853a2dfffb4f05b6f47df2280ae4e1&scope.project.id=cf3763a37383456184a9ca9351b4ce03&role.id=729f6ba7aded468788a72fefb5763fc4 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:18.007424 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b841a5a9-4377-4ea6-b584-a2ce798d604b None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:18.083333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:18.110980 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab67f680-37d4-407f-9dec-cc446ff1da4b None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:18.145916 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b841a5a9-4377-4ea6-b584-a2ce798d604b None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:18.153697 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4480d40-098c-4ad5-a665-b10b36de2322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:18.154631 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4480d40-098c-4ad5-a665-b10b36de2322 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:18.155144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4480d40-098c-4ad5-a665-b10b36de2322 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:18.155592 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4480d40-098c-4ad5-a665-b10b36de2322 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:18.157078 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4480d40-098c-4ad5-a665-b10b36de2322 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:18.157588 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4480d40-098c-4ad5-a665-b10b36de2322 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:18.159818 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4480d40-098c-4ad5-a665-b10b36de2322 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:18.170153 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 454/925] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Aug 3 04:55:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:18.177062 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4480d40-098c-4ad5-a665-b10b36de2322 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:18.190051 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aecdeb-69ff-4361-8744-b7afed9a08d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:18.190759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aecdeb-69ff-4361-8744-b7afed9a08d2 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:18.191077 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aecdeb-69ff-4361-8744-b7afed9a08d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:18.191360 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aecdeb-69ff-4361-8744-b7afed9a08d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:18.198575 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aecdeb-69ff-4361-8744-b7afed9a08d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:18.200154 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aecdeb-69ff-4361-8744-b7afed9a08d2 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:18.210085 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:18.212287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-17aecdeb-69ff-4361-8744-b7afed9a08d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:55:18.212287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:18.212287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:18.212287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:18.212287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:18.212287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:18.212287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:18.212287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:18.212287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:18.212287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:18.212287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:18.212287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:18.212287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:55:18.212287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:55:18.212287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:55:18.213179 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:55:18.213179 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:18.213179 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:18.213179 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:18.213179 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:18.213179 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:18.213179 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:18.213179 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:18.213179 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:18.213179 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:18.213179 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:18.213179 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:18.213179 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:18.213179 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:18.213179 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:18.213179 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:18.213179 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:18.214074 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:55:18.214074 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:55:18.214074 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:55:18.214074 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:55:18.214074 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:55:18.214074 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:55:18.214074 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:55:18.214074 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:18.218445 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 472/926] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:55:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:18.219392 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aecdeb-69ff-4361-8744-b7afed9a08d2 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:18.300879 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4480d40-098c-4ad5-a665-b10b36de2322 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:18.340341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0f8e0-cd55-4df3-b61b-55943efb7560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:18.341001 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0f8e0-cd55-4df3-b61b-55943efb7560 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:18.341286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0f8e0-cd55-4df3-b61b-55943efb7560 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:18.341642 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0f8e0-cd55-4df3-b61b-55943efb7560 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:18.342244 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0f8e0-cd55-4df3-b61b-55943efb7560 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:18.342552 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0f8e0-cd55-4df3-b61b-55943efb7560 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:18.343973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0f8e0-cd55-4df3-b61b-55943efb7560 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:18.349219 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aecdeb-69ff-4361-8744-b7afed9a08d2 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:18.349514 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 455/927] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Aug 3 04:55:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:18.355906 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0f8e0-cd55-4df3-b61b-55943efb7560 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:18.372657 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:18.387462 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf908fc-8d50-43bc-9f28-eab9b535ef48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:18.388277 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf908fc-8d50-43bc-9f28-eab9b535ef48 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:18.388670 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf908fc-8d50-43bc-9f28-eab9b535ef48 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:18.388978 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf908fc-8d50-43bc-9f28-eab9b535ef48 None admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:18.389704 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf908fc-8d50-43bc-9f28-eab9b535ef48 None admin] RBAC: Adding query filter params (name=service) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:18.389998 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf908fc-8d50-43bc-9f28-eab9b535ef48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:18.391647 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf908fc-8d50-43bc-9f28-eab9b535ef48 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:18.409986 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 473/928] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Aug 3 04:55:18 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:18.461943 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:18.478117 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0f8e0-cd55-4df3-b61b-55943efb7560 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:18.518148 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bcdad-c6e6-4ff0-a064-d73e20016355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:18.519524 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bcdad-c6e6-4ff0-a064-d73e20016355 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:18.519967 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bcdad-c6e6-4ff0-a064-d73e20016355 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:18.520431 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bcdad-c6e6-4ff0-a064-d73e20016355 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:18.521386 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3bcdad-c6e6-4ff0-a064-d73e20016355 None admin] RBAC: Adding query filter params (interface=public, service_id=20573b105f354d11b40712b173d013f7, region_id=RegionOne) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:18.521873 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3bcdad-c6e6-4ff0-a064-d73e20016355 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:18.523802 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3bcdad-c6e6-4ff0-a064-d73e20016355 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:18.530689 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 456/929] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Aug 3 04:55:18 2022] GET /identity/v3/endpoints?service_id=20573b105f354d11b40712b173d013f7&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:18.537042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf908fc-8d50-43bc-9f28-eab9b535ef48 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:18.613483 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4dadd-2205-4d8f-97ab-225940ad9499 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:18.613821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4dadd-2205-4d8f-97ab-225940ad9499 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:18.614033 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4dadd-2205-4d8f-97ab-225940ad9499 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:18.615847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 457/930] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:55:18.621491 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b2757-4dfa-4bee-8360-35b77ef46207 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:18.621769 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b2757-4dfa-4bee-8360-35b77ef46207 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:18.622084 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b2757-4dfa-4bee-8360-35b77ef46207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:18.644310 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-a48b2757-4dfa-4bee-8360-35b77ef46207 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:18.646187 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a48b2757-4dfa-4bee-8360-35b77ef46207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:18.651725 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf908fc-8d50-43bc-9f28-eab9b535ef48 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:18.703066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f2b97-16fc-4fde-8d6e-f78b018309d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:18.703700 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f2b97-16fc-4fde-8d6e-f78b018309d5 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:18.703984 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f2b97-16fc-4fde-8d6e-f78b018309d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:18.704260 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f2b97-16fc-4fde-8d6e-f78b018309d5 None admin] PATH_INFO: `/v3/services` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:18.704922 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f2b97-16fc-4fde-8d6e-f78b018309d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:18.707089 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f2b97-16fc-4fde-8d6e-f78b018309d5 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:18.714009 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 474/931] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Aug 3 04:55:18 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 04:55:18.744395 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 458/932] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 04:55:18.778926 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f2b97-16fc-4fde-8d6e-f78b018309d5 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:18.844730 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:18.904365 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f2b97-16fc-4fde-8d6e-f78b018309d5 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:18.940912 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfeb8d6-5035-40e6-b438-20550622da12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:18.941579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfeb8d6-5035-40e6-b438-20550622da12 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:18.941858 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfeb8d6-5035-40e6-b438-20550622da12 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:18.942137 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfeb8d6-5035-40e6-b438-20550622da12 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:18.942777 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfeb8d6-5035-40e6-b438-20550622da12 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:18.944178 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfeb8d6-5035-40e6-b438-20550622da12 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:18.953903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-8bfeb8d6-5035-40e6-b438-20550622da12 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 04:55:18.953903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:18.953903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 04:55:18.953903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 04:55:18.953903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 04:55:18.953903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 04:55:18.953903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 04:55:18.953903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:18.953903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 04:55:18.953903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:18.953903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:55:18.953903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:18.953903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:18.953903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:18.953903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:18.953903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:18.954872 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:18.954872 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:18.954872 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:18.954872 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:18.954872 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:18.954872 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:18.954872 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:18.954872 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 04:55:18.954872 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 04:55:18.954872 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 04:55:18.954872 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 04:55:18.954872 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:18.954872 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:18.954872 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:18.954872 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:18.954872 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:18.955769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:18.955769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:18.955769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:18.955769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:18.955769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:18.955769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:18.955769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:18.955769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:18.955769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:18.955769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:18.955769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:18.955769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 04:55:18.955769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:18.955769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 04:55:18.955769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:18.955769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 475/933] 10.222.0.40 () {56 vars in 1132 bytes} [Wed Aug 3 04:55:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:18.961136 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b2757-4dfa-4bee-8360-35b77ef46207 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:19.050589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:19.071570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b2757-4dfa-4bee-8360-35b77ef46207 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:19.107516 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa0e47-2d32-4fd2-81c2-58c1f1716ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:19.108531 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa0e47-2d32-4fd2-81c2-58c1f1716ebb None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:19.109033 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa0e47-2d32-4fd2-81c2-58c1f1716ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:19.109511 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa0e47-2d32-4fd2-81c2-58c1f1716ebb None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:19.110400 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa0e47-2d32-4fd2-81c2-58c1f1716ebb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:19.110930 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa0e47-2d32-4fd2-81c2-58c1f1716ebb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:19.112520 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa0e47-2d32-4fd2-81c2-58c1f1716ebb None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:19.118334 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 459/934] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Aug 3 04:55:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:19.124311 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfeb8d6-5035-40e6-b438-20550622da12 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:19.242665 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfeb8d6-5035-40e6-b438-20550622da12 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:19.252729 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:19.279093 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a24ce-c1ff-4528-9cf9-d63a6ba1c4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:19.279715 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a24ce-c1ff-4528-9cf9-d63a6ba1c4d9 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:19.280088 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a24ce-c1ff-4528-9cf9-d63a6ba1c4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:19.280360 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a24ce-c1ff-4528-9cf9-d63a6ba1c4d9 None admin] PATH_INFO: `/v3/services` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:19.281038 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a24ce-c1ff-4528-9cf9-d63a6ba1c4d9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:19.281342 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a24ce-c1ff-4528-9cf9-d63a6ba1c4d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:19.282711 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a24ce-c1ff-4528-9cf9-d63a6ba1c4d9 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:19.288311 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 476/935] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Aug 3 04:55:19 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:19.296439 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa0e47-2d32-4fd2-81c2-58c1f1716ebb None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:19.401513 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa0e47-2d32-4fd2-81c2-58c1f1716ebb None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:19.437739 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62e7bf-1688-4828-9bec-fd6faf363a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:19.438356 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62e7bf-1688-4828-9bec-fd6faf363a66 None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:19.438686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62e7bf-1688-4828-9bec-fd6faf363a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:19.438938 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62e7bf-1688-4828-9bec-fd6faf363a66 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:19.439702 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e62e7bf-1688-4828-9bec-fd6faf363a66 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:19.441117 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e62e7bf-1688-4828-9bec-fd6faf363a66 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:19.522943 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:19.529584 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 460/936] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 04:55:19 2022] POST /identity/v3/endpoints => generated 347 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:55:20.124989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b709ed-025a-4d2f-a68f-ff65ceabc43a None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:20.125515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b709ed-025a-4d2f-a68f-ff65ceabc43a None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:20.125515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b709ed-025a-4d2f-a68f-ff65ceabc43a None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:20.127148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 477/937] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:20.132504 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea30993-fde0-49e5-8c24-5b1ef3a12ba5 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:20.132798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea30993-fde0-49e5-8c24-5b1ef3a12ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:20.133072 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea30993-fde0-49e5-8c24-5b1ef3a12ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:20.156762 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-1ea30993-fde0-49e5-8c24-5b1ef3a12ba5 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:20.157960 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1ea30993-fde0-49e5-8c24-5b1ef3a12ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:20.233819 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 461/938] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:55:20.256935 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b709ed-025a-4d2f-a68f-ff65ceabc43a None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:20.375090 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b709ed-025a-4d2f-a68f-ff65ceabc43a None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:20.410908 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ba148-64bd-45dd-a271-2d8bb397d0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:20.411640 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ba148-64bd-45dd-a271-2d8bb397d0e0 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:20.411926 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ba148-64bd-45dd-a271-2d8bb397d0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:20.412208 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ba148-64bd-45dd-a271-2d8bb397d0e0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:20.424193 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742ba148-64bd-45dd-a271-2d8bb397d0e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:55:20.429213 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-742ba148-64bd-45dd-a271-2d8bb397d0e0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 04:55:20.429213 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:20.429213 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:20.429213 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:20.429213 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:20.429213 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:20.429213 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:20.429213 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:20.429213 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:20.429213 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:20.429213 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:20.429213 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:20.429213 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:55:20.429213 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:55:20.429213 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:55:20.429213 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:55:20.430333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:55:20.430333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:55:20.430333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:55:20.430333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:55:20.430333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:20.430333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:20.430333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:20.430333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:20.430333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:20.430333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:20.430333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:20.430333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:20.430333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:20.430333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:20.430333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:20.430333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:20.431403 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:20.431403 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:20.431403 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:20.431403 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:20.431403 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:55:20.431403 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:55:20.431403 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:55:20.431403 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:55:20.431403 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:55:20.431403 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:55:20.431403 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 04:55:20.431403 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:20.435085 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 478/939] 10.222.0.40 () {56 vars in 1120 bytes} [Wed Aug 3 04:55:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:20.437358 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea30993-fde0-49e5-8c24-5b1ef3a12ba5 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:20.543595 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea30993-fde0-49e5-8c24-5b1ef3a12ba5 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:20.572190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:20.581626 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ff86b-8d07-4d97-91da-96885822e562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:20.587745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ff86b-8d07-4d97-91da-96885822e562 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:20.587745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ff86b-8d07-4d97-91da-96885822e562 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:20.587745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ff86b-8d07-4d97-91da-96885822e562 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:20.587745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888ff86b-8d07-4d97-91da-96885822e562 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:20.587745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888ff86b-8d07-4d97-91da-96885822e562 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:20.587745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888ff86b-8d07-4d97-91da-96885822e562 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:20.595557 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 462/940] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Aug 3 04:55:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:20.600988 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ba148-64bd-45dd-a271-2d8bb397d0e0 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:20.635463 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:20.636795 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197af8f-41f0-4417-9960-9486dd1259a6 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:20.637095 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197af8f-41f0-4417-9960-9486dd1259a6 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:20.637380 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197af8f-41f0-4417-9960-9486dd1259a6 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:20.639639 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 463/941] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:20.644593 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0921eef-30fb-422e-8896-73f2ceb044d0 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:20.644889 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0921eef-30fb-422e-8896-73f2ceb044d0 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:20.650727 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0921eef-30fb-422e-8896-73f2ceb044d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:20.668037 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-d0921eef-30fb-422e-8896-73f2ceb044d0 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:20.670873 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d0921eef-30fb-422e-8896-73f2ceb044d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:20.724441 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ba148-64bd-45dd-a271-2d8bb397d0e0 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:20.761143 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 464/942] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:55:20.770647 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd815a4-85a3-49f2-be5c-ea9b9b97d7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:20.771664 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd815a4-85a3-49f2-be5c-ea9b9b97d7a3 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:20.772419 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd815a4-85a3-49f2-be5c-ea9b9b97d7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:20.772906 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd815a4-85a3-49f2-be5c-ea9b9b97d7a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:20.785280 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd815a4-85a3-49f2-be5c-ea9b9b97d7a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:20.786906 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0921eef-30fb-422e-8896-73f2ceb044d0 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:20.797797 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd815a4-85a3-49f2-be5c-ea9b9b97d7a3 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:20.817795 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-7cd815a4-85a3-49f2-be5c-ea9b9b97d7a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:20.817795 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:20.817795 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:20.817795 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:20.817795 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:20.817795 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:20.817795 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:20.817795 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:20.817795 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:20.817795 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:20.817795 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:20.817795 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:20.817795 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:55:20.817795 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:55:20.817795 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:55:20.818787 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:55:20.818787 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:20.818787 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:20.818787 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:20.818787 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:20.818787 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:20.818787 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:20.818787 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:20.818787 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:20.818787 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:20.818787 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:20.818787 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:20.818787 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:20.818787 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:20.818787 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:20.818787 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:20.818787 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:20.819744 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:55:20.819744 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:55:20.819744 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:20.819744 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:20.820539 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 479/943] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:20.829603 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd815a4-85a3-49f2-be5c-ea9b9b97d7a3 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:20.911616 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0921eef-30fb-422e-8896-73f2ceb044d0 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:20.921495 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:20.930519 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:20.939006 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd815a4-85a3-49f2-be5c-ea9b9b97d7a3 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:20.948712 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090dbc0-e564-42a5-b53c-4a8bba702247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:20.949340 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090dbc0-e564-42a5-b53c-4a8bba702247 None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:20.949628 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090dbc0-e564-42a5-b53c-4a8bba702247 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:20.949909 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090dbc0-e564-42a5-b53c-4a8bba702247 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:20.951181 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090dbc0-e564-42a5-b53c-4a8bba702247 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:20.952650 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090dbc0-e564-42a5-b53c-4a8bba702247 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:20.976288 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6925069-16ef-478e-87a6-f5839c9d051c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:20.976907 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6925069-16ef-478e-87a6-f5839c9d051c None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:20.977231 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6925069-16ef-478e-87a6-f5839c9d051c None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:20.977516 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6925069-16ef-478e-87a6-f5839c9d051c None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:20.978205 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6925069-16ef-478e-87a6-f5839c9d051c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:20.978511 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6925069-16ef-478e-87a6-f5839c9d051c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:20.979913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6925069-16ef-478e-87a6-f5839c9d051c None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:20.992391 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 480/944] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:55:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:20.994083 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559dc152-c9cf-4a22-8bee-9fb468965303 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:20.994083 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559dc152-c9cf-4a22-8bee-9fb468965303 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:20.994452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559dc152-c9cf-4a22-8bee-9fb468965303 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:20.996565 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 481/945] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:20.999747 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559dc152-c9cf-4a22-8bee-9fb468965303 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:21.013771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.sql.core [None req-3090dbc0-e564-42a5-b53c-4a8bba702247 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 03 04:55:21.013771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 03 04:55:21.013771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [parameters: {'user_id': 'caca6fc77da645049ff3162aa92e8478', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 03 04:55:21.013771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80355) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 03 04:55:21.020584 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-3090dbc0-e564-42a5-b53c-4a8bba702247 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 03 04:55:21.020584 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:21.020584 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 03 04:55:21.020584 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 03 04:55:21.020584 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 03 04:55:21.020584 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 03 04:55:21.020584 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 03 04:55:21.020584 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application result = self._query(query) Aug 03 04:55:21.020584 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 03 04:55:21.020584 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application conn.query(q) Aug 03 04:55:21.020584 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 03 04:55:21.020584 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 03 04:55:21.020584 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 03 04:55:21.020584 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application result.read() Aug 03 04:55:21.020584 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 03 04:55:21.021549 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 03 04:55:21.021549 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 03 04:55:21.021549 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 03 04:55:21.021549 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 03 04:55:21.021549 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 03 04:55:21.021549 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 03 04:55:21.021549 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 03 04:55:21.021549 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 03 04:55:21.021549 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:21.021549 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 03 04:55:21.021549 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:21.021549 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:21.021549 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 03 04:55:21.021549 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 03 04:55:21.021549 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 03 04:55:21.021549 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 03 04:55:21.021549 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 03 04:55:21.021549 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application next(self.gen) Aug 03 04:55:21.023234 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 03 04:55:21.023234 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application yield resource Aug 03 04:55:21.023234 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 03 04:55:21.023234 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application next(self.gen) Aug 03 04:55:21.023234 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 03 04:55:21.023234 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self.session.rollback() Aug 03 04:55:21.023234 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 03 04:55:21.023234 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self.force_reraise() Aug 03 04:55:21.023234 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 03 04:55:21.023234 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise self.value Aug 03 04:55:21.023234 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 03 04:55:21.023234 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 03 04:55:21.023234 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 03 04:55:21.023234 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application session.commit() Aug 03 04:55:21.023234 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 03 04:55:21.023234 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 03 04:55:21.023234 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 03 04:55:21.024394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._prepare_impl() Aug 03 04:55:21.024394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 03 04:55:21.024394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self.session.flush() Aug 03 04:55:21.024394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 03 04:55:21.024394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._flush(objects) Aug 03 04:55:21.024394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 03 04:55:21.024394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 03 04:55:21.024394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 03 04:55:21.024394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application compat.raise_( Aug 03 04:55:21.024394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 03 04:55:21.024394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception Aug 03 04:55:21.024394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 03 04:55:21.024394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application flush_context.execute() Aug 03 04:55:21.024394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 03 04:55:21.024394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rec.execute(self) Aug 03 04:55:21.024394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 03 04:55:21.024394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 03 04:55:21.025500 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 03 04:55:21.025500 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 03 04:55:21.025500 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 03 04:55:21.025500 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 03 04:55:21.025500 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 03 04:55:21.025500 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 03 04:55:21.025500 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 03 04:55:21.025500 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 03 04:55:21.025500 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 03 04:55:21.025500 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 03 04:55:21.025500 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 03 04:55:21.025500 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 03 04:55:21.025500 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 03 04:55:21.025500 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 03 04:55:21.025500 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 03 04:55:21.025500 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception Aug 03 04:55:21.026550 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 03 04:55:21.026550 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 03 04:55:21.026550 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 03 04:55:21.026550 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 03 04:55:21.026550 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 03 04:55:21.026550 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application result = self._query(query) Aug 03 04:55:21.026550 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 03 04:55:21.026550 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application conn.query(q) Aug 03 04:55:21.026550 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 03 04:55:21.026550 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 03 04:55:21.026550 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 03 04:55:21.026550 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application result.read() Aug 03 04:55:21.026550 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 03 04:55:21.026550 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 03 04:55:21.026550 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 03 04:55:21.026550 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 03 04:55:21.029423 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 03 04:55:21.029423 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 03 04:55:21.029423 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 03 04:55:21.029423 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 03 04:55:21.029423 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 03 04:55:21.029423 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 03 04:55:21.029423 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [parameters: {'user_id': 'caca6fc77da645049ff3162aa92e8478', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 03 04:55:21.029423 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 03 04:55:21.029423 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:21.029423 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:55:21.029423 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:21.029423 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:21.029423 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:21.029423 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:21.029423 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:21.031635 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:21.031635 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:21.031635 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:21.031635 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:21.031635 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:21.031635 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:21.031635 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:21.031635 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 03 04:55:21.031635 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 03 04:55:21.031635 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:21.031635 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:21.031635 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:55:21.031635 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:55:21.031635 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:55:21.031635 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:55:21.031635 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 03 04:55:21.031635 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 03 04:55:21.035336 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 03 04:55:21.035336 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 03 04:55:21.035336 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 03 04:55:21.035336 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 03 04:55:21.035336 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 03 04:55:21.035336 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:21.035336 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 465/946] 10.222.0.40 () {60 vars in 1131 bytes} [Wed Aug 3 04:55:20 2022] POST /identity/v3/users => generated 160 bytes in 236 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:55:21.035336 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472c1ac-c15d-4d31-8418-d57f81bcbe05 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:21.035336 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472c1ac-c15d-4d31-8418-d57f81bcbe05 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:21.035336 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472c1ac-c15d-4d31-8418-d57f81bcbe05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:21.046271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-0472c1ac-c15d-4d31-8418-d57f81bcbe05 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:21.047856 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0472c1ac-c15d-4d31-8418-d57f81bcbe05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:21.132839 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559dc152-c9cf-4a22-8bee-9fb468965303 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:21.134941 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 466/947] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:55:21.134941 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472c1ac-c15d-4d31-8418-d57f81bcbe05 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:21.192491 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968365ec-e99c-4683-b87b-1bbca5c5f154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:21.193428 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968365ec-e99c-4683-b87b-1bbca5c5f154 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:21.193716 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968365ec-e99c-4683-b87b-1bbca5c5f154 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:21.194014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968365ec-e99c-4683-b87b-1bbca5c5f154 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:21.203671 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968365ec-e99c-4683-b87b-1bbca5c5f154 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:21.205143 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968365ec-e99c-4683-b87b-1bbca5c5f154 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:21.222708 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-968365ec-e99c-4683-b87b-1bbca5c5f154 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 04:55:21.222708 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:21.222708 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:21.222708 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:21.222708 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:21.222708 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:21.222708 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:21.222708 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:21.222708 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:21.222708 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:21.222708 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:21.222708 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:21.222708 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:55:21.222708 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:55:21.222708 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:55:21.222708 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:55:21.224243 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:21.224243 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:21.224243 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:55:21.224243 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:55:21.224243 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:55:21.224243 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:55:21.224243 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:21.224243 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:21.224243 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:21.224243 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:21.224243 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:21.224243 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:21.224243 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:21.224243 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:21.224243 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:21.224243 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:21.224243 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:21.224243 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:21.225449 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:21.225449 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:21.225449 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:55:21.225449 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:55:21.225449 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:55:21.225449 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:55:21.225449 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:55:21.225449 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:55:21.225449 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 04:55:21.225449 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:21.228780 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 482/948] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Aug 3 04:55:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:21.229572 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968365ec-e99c-4683-b87b-1bbca5c5f154 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:21.233579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:21.266830 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472c1ac-c15d-4d31-8418-d57f81bcbe05 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:21.277796 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:21.302929 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36b1b7-e7a1-490a-949b-30b587dbc487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:21.304109 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36b1b7-e7a1-490a-949b-30b587dbc487 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:21.304705 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36b1b7-e7a1-490a-949b-30b587dbc487 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:21.305615 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36b1b7-e7a1-490a-949b-30b587dbc487 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:21.315111 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e36b1b7-e7a1-490a-949b-30b587dbc487 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:21.317131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e36b1b7-e7a1-490a-949b-30b587dbc487 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:21.329571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-1e36b1b7-e7a1-490a-949b-30b587dbc487 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 04:55:21.329571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:21.329571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:21.329571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:21.329571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:21.329571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:21.329571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:21.329571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:21.329571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:21.329571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:21.329571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:21.329571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:21.329571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:55:21.329571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:55:21.329571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:55:21.329571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:55:21.330748 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:21.330748 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:21.330748 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:55:21.330748 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:55:21.330748 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:55:21.330748 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:55:21.330748 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:21.330748 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:21.330748 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:21.330748 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:21.330748 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:21.330748 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:21.330748 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:21.330748 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:21.330748 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:21.330748 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:21.330748 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:21.330748 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:21.331803 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:21.331803 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:21.331803 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:55:21.331803 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:55:21.331803 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:55:21.331803 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:55:21.331803 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:55:21.331803 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:55:21.331803 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 04:55:21.331803 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:21.336046 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 467/949] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:55:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:21.336841 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36b1b7-e7a1-490a-949b-30b587dbc487 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:21.345628 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968365ec-e99c-4683-b87b-1bbca5c5f154 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:21.384811 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4b006-4a37-40eb-b578-f340fb9b2bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:21.385457 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4b006-4a37-40eb-b578-f340fb9b2bda None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:21.385744 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4b006-4a37-40eb-b578-f340fb9b2bda None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:21.386023 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4b006-4a37-40eb-b578-f340fb9b2bda None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:21.386652 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d4b006-4a37-40eb-b578-f340fb9b2bda None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:21.388731 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d4b006-4a37-40eb-b578-f340fb9b2bda None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:21.398590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-70d4b006-4a37-40eb-b578-f340fb9b2bda None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 04:55:21.398590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:21.398590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 04:55:21.398590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 04:55:21.398590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 04:55:21.398590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 04:55:21.398590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 04:55:21.398590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:21.398590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 04:55:21.398590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:21.398590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:55:21.398590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:21.398590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:21.398590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:21.398590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:21.398590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:21.398590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:21.400076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:21.400076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:21.400076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:21.400076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:21.400076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:21.400076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:21.400076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 04:55:21.400076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 04:55:21.400076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 04:55:21.400076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 04:55:21.400076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:21.400076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:21.400076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:21.400076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:21.400076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:21.400076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:21.401035 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:21.401035 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:21.401035 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:21.401035 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:21.401035 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:21.401035 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:21.401035 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:21.401035 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:21.401035 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:21.401035 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:21.401035 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 04:55:21.401035 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:21.401035 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 04:55:21.401035 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:21.401035 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 483/950] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Aug 3 04:55:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:21.401843 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4b006-4a37-40eb-b578-f340fb9b2bda None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:21.452730 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36b1b7-e7a1-490a-949b-30b587dbc487 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:21.479916 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:21.490390 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18661899-759b-4f31-a976-9f0e0b447c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:21.491077 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18661899-759b-4f31-a976-9f0e0b447c95 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:21.491739 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18661899-759b-4f31-a976-9f0e0b447c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:21.491739 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18661899-759b-4f31-a976-9f0e0b447c95 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:21.492421 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18661899-759b-4f31-a976-9f0e0b447c95 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:21.492732 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18661899-759b-4f31-a976-9f0e0b447c95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:21.494277 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18661899-759b-4f31-a976-9f0e0b447c95 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:21.519212 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 468/951] 10.222.0.40 () {56 vars in 1149 bytes} [Wed Aug 3 04:55:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:21.519212 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18661899-759b-4f31-a976-9f0e0b447c95 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:21.522736 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4b006-4a37-40eb-b578-f340fb9b2bda None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:21.554890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:21.564554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14e67a-d6f7-4557-b446-68ac2bdd24ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:21.566328 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14e67a-d6f7-4557-b446-68ac2bdd24ca None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:21.566328 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14e67a-d6f7-4557-b446-68ac2bdd24ca None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:21.566328 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14e67a-d6f7-4557-b446-68ac2bdd24ca None admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:21.566575 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14e67a-d6f7-4557-b446-68ac2bdd24ca None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:21.566755 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14e67a-d6f7-4557-b446-68ac2bdd24ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:21.568241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14e67a-d6f7-4557-b446-68ac2bdd24ca None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:21.611377 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 484/952] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Aug 3 04:55:21 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:21.612388 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14e67a-d6f7-4557-b446-68ac2bdd24ca None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:21.650135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18661899-759b-4f31-a976-9f0e0b447c95 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:21.692394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab2b47-9106-4647-bb3f-888b35b435a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:21.693470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab2b47-9106-4647-bb3f-888b35b435a1 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:21.693958 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab2b47-9106-4647-bb3f-888b35b435a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:21.694457 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab2b47-9106-4647-bb3f-888b35b435a1 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:21.695412 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dab2b47-9106-4647-bb3f-888b35b435a1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:21.695946 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dab2b47-9106-4647-bb3f-888b35b435a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:21.697416 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dab2b47-9106-4647-bb3f-888b35b435a1 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:21.704112 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 469/953] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Aug 3 04:55:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:21.705532 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34c10b-498d-4324-ba42-5ff728b34a4e None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:21.705900 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34c10b-498d-4324-ba42-5ff728b34a4e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:21.706252 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34c10b-498d-4324-ba42-5ff728b34a4e None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:21.707939 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 470/954] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:21.713424 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331d43b-2b1a-48dd-afb6-7df420a4ce55 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:21.713712 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331d43b-2b1a-48dd-afb6-7df420a4ce55 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:21.713994 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331d43b-2b1a-48dd-afb6-7df420a4ce55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:21.738026 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-8331d43b-2b1a-48dd-afb6-7df420a4ce55 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:21.739620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8331d43b-2b1a-48dd-afb6-7df420a4ce55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:21.743667 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14e67a-d6f7-4557-b446-68ac2bdd24ca None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:21.772727 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:21.781738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6e460-0c71-48cb-8893-55bca1ff3a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:21.782391 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6e460-0c71-48cb-8893-55bca1ff3a9e None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:21.784436 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6e460-0c71-48cb-8893-55bca1ff3a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:21.784436 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6e460-0c71-48cb-8893-55bca1ff3a9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:21.794824 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6e460-0c71-48cb-8893-55bca1ff3a9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:21.796870 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6e460-0c71-48cb-8893-55bca1ff3a9e None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:21.814361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-74c6e460-0c71-48cb-8893-55bca1ff3a9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:21.814361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:21.814361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:21.814361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:21.814361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:21.814361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:21.814361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:21.814361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:21.814361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:21.814361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:21.814361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:21.814361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:21.814361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:55:21.814361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:55:21.814361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:55:21.815373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:55:21.815373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:21.815373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:21.815373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:21.815373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:21.815373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:21.815373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:21.815373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:21.815373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:21.815373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:21.815373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:21.815373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:21.815373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:21.815373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:21.815373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:21.815373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:21.815373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:21.816278 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:55:21.816278 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:55:21.816278 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:21.816278 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:21.816278 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 485/955] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:21.816557 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6e460-0c71-48cb-8893-55bca1ff3a9e None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:21.833980 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 471/956] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 04:55:21.835330 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8331d43b-2b1a-48dd-afb6-7df420a4ce55 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:21.865831 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:21.944099 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6e460-0c71-48cb-8893-55bca1ff3a9e None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:21.947087 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8331d43b-2b1a-48dd-afb6-7df420a4ce55 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:21.978514 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df2f6d-fc19-43e1-a5a5-4d15204855ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:21.979111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df2f6d-fc19-43e1-a5a5-4d15204855ca None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:21.979397 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df2f6d-fc19-43e1-a5a5-4d15204855ca None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:21.979697 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df2f6d-fc19-43e1-a5a5-4d15204855ca None admin] PATH_INFO: `/v3/services` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:21.980366 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df2f6d-fc19-43e1-a5a5-4d15204855ca None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:21.980670 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df2f6d-fc19-43e1-a5a5-4d15204855ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:21.982083 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d0122-3f66-4d0b-ac49-283d95f82ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:21.982219 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df2f6d-fc19-43e1-a5a5-4d15204855ca None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:21.982688 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d0122-3f66-4d0b-ac49-283d95f82ecb None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:21.983002 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d0122-3f66-4d0b-ac49-283d95f82ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:21.983276 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d0122-3f66-4d0b-ac49-283d95f82ecb None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:21.983965 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706d0122-3f66-4d0b-ac49-283d95f82ecb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:21.984250 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706d0122-3f66-4d0b-ac49-283d95f82ecb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:21.985588 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706d0122-3f66-4d0b-ac49-283d95f82ecb None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:21.986924 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 486/957] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Aug 3 04:55:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:21.987777 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df2f6d-fc19-43e1-a5a5-4d15204855ca None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:22.001686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 472/958] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:22.001686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d0122-3f66-4d0b-ac49-283d95f82ecb None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:22.037801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:22.079525 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:22.094939 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df2f6d-fc19-43e1-a5a5-4d15204855ca None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:22.097858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d0122-3f66-4d0b-ac49-283d95f82ecb None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:22.128317 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d2816-027f-45d4-a4f5-b9e438af748e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:22.128964 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d2816-027f-45d4-a4f5-b9e438af748e None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:22.129919 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d2816-027f-45d4-a4f5-b9e438af748e None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:22.129919 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d2816-027f-45d4-a4f5-b9e438af748e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:22.130160 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16d2816-027f-45d4-a4f5-b9e438af748e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:22.131575 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16d2816-027f-45d4-a4f5-b9e438af748e None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:22.132837 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e40f02b-2e5c-452f-a305-f36c728c2d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:22.133461 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e40f02b-2e5c-452f-a305-f36c728c2d1d None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:22.133746 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e40f02b-2e5c-452f-a305-f36c728c2d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:22.134026 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e40f02b-2e5c-452f-a305-f36c728c2d1d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:22.134745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e40f02b-2e5c-452f-a305-f36c728c2d1d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:22.136135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e40f02b-2e5c-452f-a305-f36c728c2d1d None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:22.141368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-b16d2816-027f-45d4-a4f5-b9e438af748e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 04:55:22.141368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:22.141368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 04:55:22.141368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 04:55:22.141368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 04:55:22.141368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 04:55:22.141368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 04:55:22.141368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:22.141368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 04:55:22.141368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:22.141368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:55:22.141368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:22.141368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:22.141368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:22.141368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:22.141368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:22.141368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:22.142678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:22.142678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:22.142678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:22.142678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:22.142678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:22.142678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:22.142678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 04:55:22.142678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 04:55:22.142678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 04:55:22.142678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 04:55:22.142678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:22.142678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:22.142678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:22.142678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:22.142678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:22.142678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:22.143582 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:22.143582 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:22.143582 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:22.143582 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:22.143582 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:22.143582 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:22.143582 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:22.143582 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:22.143582 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:22.143582 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:22.143582 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 04:55:22.143582 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:22.143582 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 04:55:22.143582 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:22.143582 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 487/959] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:55:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:22.144523 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d2816-027f-45d4-a4f5-b9e438af748e None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:22.219937 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 473/960] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 04:55:21 2022] POST /identity/v3/endpoints => generated 349 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:55:22.220745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e40f02b-2e5c-452f-a305-f36c728c2d1d None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:22.249132 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d2816-027f-45d4-a4f5-b9e438af748e None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:22.257716 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:22.297192 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403c52f-5b89-4875-bfde-61a33e117edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:22.298204 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403c52f-5b89-4875-bfde-61a33e117edb None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:22.298496 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403c52f-5b89-4875-bfde-61a33e117edb None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:22.298780 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403c52f-5b89-4875-bfde-61a33e117edb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:22.306510 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4403c52f-5b89-4875-bfde-61a33e117edb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:22.308080 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4403c52f-5b89-4875-bfde-61a33e117edb None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:22.324263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-4403c52f-5b89-4875-bfde-61a33e117edb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:55:22.324263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:22.324263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:22.324263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:22.324263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:22.324263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:22.324263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:22.324263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:22.324263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:22.324263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:22.324263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:22.324263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:22.324263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:55:22.324263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:55:22.324263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:55:22.326059 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:55:22.326059 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:22.326059 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:22.326059 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:22.326059 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:22.326059 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:22.326059 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:22.326059 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:22.326059 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:22.326059 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:22.326059 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:22.326059 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:22.326059 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:22.326059 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:22.326059 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:22.326059 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:22.326059 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:22.327208 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:55:22.327208 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:55:22.327208 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:55:22.327208 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:55:22.327208 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:55:22.327208 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:55:22.327208 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:55:22.327208 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:22.329052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 488/961] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:55:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:22.335336 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403c52f-5b89-4875-bfde-61a33e117edb None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:22.364023 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e40f02b-2e5c-452f-a305-f36c728c2d1d None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:22.404325 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e6c9c-664b-408b-b1f6-dd28ba9682c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:22.405904 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e6c9c-664b-408b-b1f6-dd28ba9682c9 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:22.405904 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e6c9c-664b-408b-b1f6-dd28ba9682c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:22.405904 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e6c9c-664b-408b-b1f6-dd28ba9682c9 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:22.406182 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e6c9c-664b-408b-b1f6-dd28ba9682c9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:22.406474 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e6c9c-664b-408b-b1f6-dd28ba9682c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:22.407915 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e6c9c-664b-408b-b1f6-dd28ba9682c9 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:22.414273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 474/962] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Aug 3 04:55:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:22.420314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e6c9c-664b-408b-b1f6-dd28ba9682c9 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:22.423160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:22.471614 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403c52f-5b89-4875-bfde-61a33e117edb None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:22.508572 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb114ae2-e1ac-42f0-aff2-defbbd5928a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:22.509115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb114ae2-e1ac-42f0-aff2-defbbd5928a2 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:22.509399 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb114ae2-e1ac-42f0-aff2-defbbd5928a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:22.509681 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb114ae2-e1ac-42f0-aff2-defbbd5928a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:22.510545 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb114ae2-e1ac-42f0-aff2-defbbd5928a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:22.510751 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb114ae2-e1ac-42f0-aff2-defbbd5928a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:22.512230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb114ae2-e1ac-42f0-aff2-defbbd5928a2 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:22.525634 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 489/963] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:55:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:22.529447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e6c9c-664b-408b-b1f6-dd28ba9682c9 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:22.531423 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb114ae2-e1ac-42f0-aff2-defbbd5928a2 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:22.558376 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:22.566869 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1ac3c-6d5b-4e10-956a-570650aa8b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:22.567539 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1ac3c-6d5b-4e10-956a-570650aa8b45 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:22.580051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1ac3c-6d5b-4e10-956a-570650aa8b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:22.580051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1ac3c-6d5b-4e10-956a-570650aa8b45 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:22.580051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1ac3c-6d5b-4e10-956a-570650aa8b45 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:22.580051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1ac3c-6d5b-4e10-956a-570650aa8b45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:22.580051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1ac3c-6d5b-4e10-956a-570650aa8b45 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:22.580051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 475/964] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Aug 3 04:55:22 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:22.646107 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb114ae2-e1ac-42f0-aff2-defbbd5928a2 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:22.689672 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6dc530-2a74-492a-baf2-bb913aa985b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:22.690373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6dc530-2a74-492a-baf2-bb913aa985b3 None admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:22.690661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6dc530-2a74-492a-baf2-bb913aa985b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:22.690977 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6dc530-2a74-492a-baf2-bb913aa985b3 None admin] PATH_INFO: `/v3/projects/cf3763a37383456184a9ca9351b4ce03/users/de853a2dfffb4f05b6f47df2280ae4e1/roles/729f6ba7aded468788a72fefb5763fc4` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:22.693409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6dc530-2a74-492a-baf2-bb913aa985b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf3763a37383456184a9ca9351b4ce03, user_id=de853a2dfffb4f05b6f47df2280ae4e1, role_id=729f6ba7aded468788a72fefb5763fc4)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:22.694927 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6dc530-2a74-492a-baf2-bb913aa985b3 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:22.796433 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 490/965] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 3 04:55:22 2022] PUT /identity/v3/projects/cf3763a37383456184a9ca9351b4ce03/users/de853a2dfffb4f05b6f47df2280ae4e1/roles/729f6ba7aded468788a72fefb5763fc4 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 04:55:24.258684 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07d013-94e4-44d0-8415-7344836dd88f None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:24.259441 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07d013-94e4-44d0-8415-7344836dd88f None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:24.259441 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07d013-94e4-44d0-8415-7344836dd88f None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:24.260645 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 476/966] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:24.266281 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6ffa8-afc9-4dc5-9aa9-b7397a887fa3 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:24.266698 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6ffa8-afc9-4dc5-9aa9-b7397a887fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:24.266956 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6ffa8-afc9-4dc5-9aa9-b7397a887fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:24.285597 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-a8f6ffa8-afc9-4dc5-9aa9-b7397a887fa3 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:24.286792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-a8f6ffa8-afc9-4dc5-9aa9-b7397a887fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:24.354868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 491/967] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:55:24.386953 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07d013-94e4-44d0-8415-7344836dd88f None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:24.455522 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:24.492723 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07d013-94e4-44d0-8415-7344836dd88f None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:24.539995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c762226a-7df1-4584-93b4-c9157d51aed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:24.540555 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762226a-7df1-4584-93b4-c9157d51aed9 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:24.540842 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762226a-7df1-4584-93b4-c9157d51aed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:24.541117 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762226a-7df1-4584-93b4-c9157d51aed9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:24.541739 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c762226a-7df1-4584-93b4-c9157d51aed9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:24.543431 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c762226a-7df1-4584-93b4-c9157d51aed9 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:24.553873 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-c762226a-7df1-4584-93b4-c9157d51aed9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 04:55:24.553873 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:24.553873 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 04:55:24.553873 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 04:55:24.553873 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 04:55:24.553873 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 04:55:24.553873 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 04:55:24.553873 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:24.553873 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 04:55:24.553873 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:24.553873 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:55:24.553873 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:24.553873 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:24.553873 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:24.553873 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:24.553873 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:24.553873 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:24.555015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:24.555015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:24.555015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:24.555015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:24.555015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:24.555015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:24.555015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 04:55:24.555015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 04:55:24.555015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 04:55:24.555015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 04:55:24.555015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:24.555015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:24.555015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:24.555015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:24.555015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:24.555015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:24.555951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:24.555951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:24.555951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:24.555951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:24.555951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:24.555951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:24.555951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:24.555951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:24.555951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:24.555951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:24.555951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 04:55:24.555951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:24.555951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 04:55:24.555951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:24.559696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 477/968] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Aug 3 04:55:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:24.561579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6ffa8-afc9-4dc5-9aa9-b7397a887fa3 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:24.577117 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:24.616814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194fec6-a033-4e94-98a5-92ff69fa2479 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:24.617163 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194fec6-a033-4e94-98a5-92ff69fa2479 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:24.617448 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194fec6-a033-4e94-98a5-92ff69fa2479 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:24.618878 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 478/969] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:55:24.624143 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df2599-21a6-451f-986d-58163546b369 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:24.624438 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df2599-21a6-451f-986d-58163546b369 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:24.624720 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df2599-21a6-451f-986d-58163546b369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:24.646262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-36df2599-21a6-451f-986d-58163546b369 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:24.647749 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-36df2599-21a6-451f-986d-58163546b369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:24.675895 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6ffa8-afc9-4dc5-9aa9-b7397a887fa3 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:24.712746 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb1ea6d-0309-44c8-92aa-9aec77267448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:24.713366 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1ea6d-0309-44c8-92aa-9aec77267448 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:24.713652 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1ea6d-0309-44c8-92aa-9aec77267448 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:24.713924 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1ea6d-0309-44c8-92aa-9aec77267448 None admin] PATH_INFO: `/v3/services` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:24.714597 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb1ea6d-0309-44c8-92aa-9aec77267448 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:24.714925 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb1ea6d-0309-44c8-92aa-9aec77267448 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:24.716348 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb1ea6d-0309-44c8-92aa-9aec77267448 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:24.721137 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 479/970] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 04:55:24.721504 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 492/971] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Aug 3 04:55:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:24.726900 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df2599-21a6-451f-986d-58163546b369 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:24.744835 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb1ea6d-0309-44c8-92aa-9aec77267448 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:24.791578 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:24.829029 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:24.844033 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df2599-21a6-451f-986d-58163546b369 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:24.860549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb1ea6d-0309-44c8-92aa-9aec77267448 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:24.879518 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6e985-4a9d-4cab-b5e2-da454838e7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:24.880145 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6e985-4a9d-4cab-b5e2-da454838e7af None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:24.880430 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6e985-4a9d-4cab-b5e2-da454838e7af None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:24.880713 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6e985-4a9d-4cab-b5e2-da454838e7af None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:24.881396 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c6e985-4a9d-4cab-b5e2-da454838e7af None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:24.881706 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c6e985-4a9d-4cab-b5e2-da454838e7af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:24.883122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c6e985-4a9d-4cab-b5e2-da454838e7af None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:24.888539 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 480/972] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Aug 3 04:55:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:24.889762 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d253d1-cc64-4f9a-8996-64a617ae43e3 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:24.890057 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d253d1-cc64-4f9a-8996-64a617ae43e3 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:24.890339 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d253d1-cc64-4f9a-8996-64a617ae43e3 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:24.891743 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 481/973] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:24.894121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d253d1-cc64-4f9a-8996-64a617ae43e3 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:24.900819 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67da3cc-e0cf-4547-96e9-55762dc415dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:24.901445 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67da3cc-e0cf-4547-96e9-55762dc415dd None admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:24.901732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67da3cc-e0cf-4547-96e9-55762dc415dd None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:24.902011 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67da3cc-e0cf-4547-96e9-55762dc415dd None admin] PATH_INFO: `/v3/services` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:24.902711 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67da3cc-e0cf-4547-96e9-55762dc415dd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:24.904130 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67da3cc-e0cf-4547-96e9-55762dc415dd None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:24.967274 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 493/974] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Aug 3 04:55:24 2022] POST /identity/v3/services => generated 242 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 04:55:24.968539 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43737821-9dcd-41d4-9985-f7d5aa5bf263 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:24.968804 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43737821-9dcd-41d4-9985-f7d5aa5bf263 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:24.969081 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43737821-9dcd-41d4-9985-f7d5aa5bf263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:24.992450 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-43737821-9dcd-41d4-9985-f7d5aa5bf263 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:24.993709 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-43737821-9dcd-41d4-9985-f7d5aa5bf263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:25.010461 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d253d1-cc64-4f9a-8996-64a617ae43e3 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:25.036307 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:25.053665 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b0297d-889e-4b8a-bb52-9dfc3a6755d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:25.054260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0297d-889e-4b8a-bb52-9dfc3a6755d6 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:25.054536 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0297d-889e-4b8a-bb52-9dfc3a6755d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:25.054813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0297d-889e-4b8a-bb52-9dfc3a6755d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:25.055624 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b0297d-889e-4b8a-bb52-9dfc3a6755d6 None admin] RBAC: Adding query filter params (interface=admin, service_id=20573b105f354d11b40712b173d013f7, region_id=RegionOne) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:25.055938 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b0297d-889e-4b8a-bb52-9dfc3a6755d6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:25.057456 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b0297d-889e-4b8a-bb52-9dfc3a6755d6 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:25.066431 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 482/975] 10.222.0.40 () {56 vars in 1249 bytes} [Wed Aug 3 04:55:24 2022] GET /identity/v3/endpoints?service_id=20573b105f354d11b40712b173d013f7&interface=admin®ion_id=RegionOne => generated 193 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:25.072228 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b0297d-889e-4b8a-bb52-9dfc3a6755d6 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:25.075964 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 494/976] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:24 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:55:25.101380 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43737821-9dcd-41d4-9985-f7d5aa5bf263 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:25.177173 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:25.212554 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b0297d-889e-4b8a-bb52-9dfc3a6755d6 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:25.222702 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43737821-9dcd-41d4-9985-f7d5aa5bf263 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:25.250273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7a507-342f-4531-87eb-b2b375aaa104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:25.250915 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7a507-342f-4531-87eb-b2b375aaa104 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:25.251202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7a507-342f-4531-87eb-b2b375aaa104 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:25.251481 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7a507-342f-4531-87eb-b2b375aaa104 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:25.252121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7a507-342f-4531-87eb-b2b375aaa104 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:25.253479 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7a507-342f-4531-87eb-b2b375aaa104 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:25.259411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52013dbc-968b-4808-8d1f-b09caf29ee13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:25.266864 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52013dbc-968b-4808-8d1f-b09caf29ee13 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:25.267526 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 483/977] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Aug 3 04:55:25 2022] GET /identity/v3/services => generated 1727 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 04:55:25.267880 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52013dbc-968b-4808-8d1f-b09caf29ee13 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:25.267880 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52013dbc-968b-4808-8d1f-b09caf29ee13 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:25.279490 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52013dbc-968b-4808-8d1f-b09caf29ee13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:55:25.284799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-52013dbc-968b-4808-8d1f-b09caf29ee13 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 04:55:25.284799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:25.284799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:25.284799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:25.284799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:25.284799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:25.284799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:25.284799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:25.284799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:25.284799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:25.284799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:25.284799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:25.284799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:55:25.284799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:55:25.284799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:55:25.284799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:55:25.285832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:55:25.285832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:55:25.285832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:55:25.285832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:55:25.285832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:25.285832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:25.285832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:25.285832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:25.285832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:25.285832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:25.285832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:25.285832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:25.285832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:25.285832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:25.285832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:25.285832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:25.286705 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:25.286705 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:25.286705 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:25.286705 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:25.286705 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:55:25.286705 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:55:25.286705 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:55:25.286705 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:55:25.286705 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:55:25.286705 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:55:25.286705 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 04:55:25.286705 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:25.291260 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 495/978] 10.222.0.40 () {56 vars in 1120 bytes} [Wed Aug 3 04:55:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:55:25.292472 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7a507-342f-4531-87eb-b2b375aaa104 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:25.349214 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:25.406609 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7a507-342f-4531-87eb-b2b375aaa104 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:25.444989 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06373f4-7f6e-4a51-978c-fcbdcfc2352a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:25.445615 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06373f4-7f6e-4a51-978c-fcbdcfc2352a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:25.445902 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06373f4-7f6e-4a51-978c-fcbdcfc2352a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:25.446179 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06373f4-7f6e-4a51-978c-fcbdcfc2352a None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:25.446943 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06373f4-7f6e-4a51-978c-fcbdcfc2352a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:25.447251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06373f4-7f6e-4a51-978c-fcbdcfc2352a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:25.448626 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06373f4-7f6e-4a51-978c-fcbdcfc2352a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:25.458284 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 484/979] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Aug 3 04:55:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:25.464187 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52013dbc-968b-4808-8d1f-b09caf29ee13 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:25.570153 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52013dbc-968b-4808-8d1f-b09caf29ee13 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:25.596292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:25.606956 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e900d-9c7e-44a3-bd38-755110d89bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:25.606956 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e900d-9c7e-44a3-bd38-755110d89bca None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:25.607238 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e900d-9c7e-44a3-bd38-755110d89bca None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:25.607482 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e900d-9c7e-44a3-bd38-755110d89bca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:25.618287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e900d-9c7e-44a3-bd38-755110d89bca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:25.619800 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e900d-9c7e-44a3-bd38-755110d89bca None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:25.635128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-8f2e900d-9c7e-44a3-bd38-755110d89bca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:25.635128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:25.635128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:25.635128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:25.635128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:25.635128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:25.635128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:25.635128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:25.635128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:25.635128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:25.635128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:25.635128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:25.635128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:55:25.635128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:55:25.635128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:55:25.636128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:55:25.636128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:25.636128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:25.636128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:25.636128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:25.636128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:25.636128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:25.636128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:25.636128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:25.636128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:25.636128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:25.636128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:25.636128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:25.636128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:25.636128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:25.636128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:25.636128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:25.641888 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:55:25.641888 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:55:25.641888 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:25.641888 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:25.641888 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 496/980] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:55:25.643845 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06373f4-7f6e-4a51-978c-fcbdcfc2352a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:25.755461 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06373f4-7f6e-4a51-978c-fcbdcfc2352a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:25.766522 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:25.790628 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182cbdf6-8b47-4a93-af64-a0ed0697d68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:25.791281 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182cbdf6-8b47-4a93-af64-a0ed0697d68b None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:25.791569 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182cbdf6-8b47-4a93-af64-a0ed0697d68b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:25.791847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182cbdf6-8b47-4a93-af64-a0ed0697d68b None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:25.792534 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182cbdf6-8b47-4a93-af64-a0ed0697d68b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:25.792844 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182cbdf6-8b47-4a93-af64-a0ed0697d68b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:25.794287 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182cbdf6-8b47-4a93-af64-a0ed0697d68b None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:25.807900 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 485/981] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:25.813406 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e900d-9c7e-44a3-bd38-755110d89bca None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:25.923358 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e900d-9c7e-44a3-bd38-755110d89bca None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:25.959655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4455ab-6e65-43f5-9848-c27da40459f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:25.960304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4455ab-6e65-43f5-9848-c27da40459f0 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:25.960591 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4455ab-6e65-43f5-9848-c27da40459f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:25.960869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4455ab-6e65-43f5-9848-c27da40459f0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:25.968960 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4455ab-6e65-43f5-9848-c27da40459f0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:25.970430 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4455ab-6e65-43f5-9848-c27da40459f0 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:25.982452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-6e4455ab-6e65-43f5-9848-c27da40459f0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 04:55:25.982452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:25.982452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:25.982452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:25.982452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:25.982452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:25.982452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:25.982452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:25.982452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:25.982452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:25.982452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:25.982452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:25.982452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:55:25.982452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:55:25.982452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:55:25.982452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:55:25.983456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:25.983456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:25.983456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:55:25.983456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:55:25.983456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:55:25.983456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:55:25.983456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:25.983456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:25.983456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:25.983456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:25.983456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:25.983456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:25.983456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:25.983456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:25.983456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:25.983456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:25.983456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:25.983456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:25.989022 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:25.989022 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:25.989022 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:55:25.989022 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:55:25.989022 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:55:25.989022 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:55:25.989022 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:55:25.989022 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:55:25.989022 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 04:55:25.989022 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:25.989022 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 497/982] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Aug 3 04:55:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:55:25.991515 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182cbdf6-8b47-4a93-af64-a0ed0697d68b None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:26.095409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182cbdf6-8b47-4a93-af64-a0ed0697d68b None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:26.131214 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d3aed-7d4d-43e9-97e0-f3d6069fd1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:26.131828 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d3aed-7d4d-43e9-97e0-f3d6069fd1ba None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:26.132108 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d3aed-7d4d-43e9-97e0-f3d6069fd1ba None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:26.132380 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d3aed-7d4d-43e9-97e0-f3d6069fd1ba None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:26.133164 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d3aed-7d4d-43e9-97e0-f3d6069fd1ba None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:26.133466 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d3aed-7d4d-43e9-97e0-f3d6069fd1ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:26.134902 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d3aed-7d4d-43e9-97e0-f3d6069fd1ba None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:26.157295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 486/983] 10.222.0.40 () {56 vars in 1149 bytes} [Wed Aug 3 04:55:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:26.158565 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4455ab-6e65-43f5-9848-c27da40459f0 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:26.201446 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:26.261312 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4455ab-6e65-43f5-9848-c27da40459f0 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:26.295391 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03b215-a5ca-4533-8acb-33c88f95b5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:26.296007 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03b215-a5ca-4533-8acb-33c88f95b5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:26.296271 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03b215-a5ca-4533-8acb-33c88f95b5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:26.296544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03b215-a5ca-4533-8acb-33c88f95b5e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:26.368841 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03b215-a5ca-4533-8acb-33c88f95b5e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:26.370310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03b215-a5ca-4533-8acb-33c88f95b5e6 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:26.425778 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:26.490053 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 498/984] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:55:26 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 03 04:55:26.495421 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d3aed-7d4d-43e9-97e0-f3d6069fd1ba None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:26.525271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:26.598561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d3aed-7d4d-43e9-97e0-f3d6069fd1ba None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:26.632018 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73957374-9ffe-4b8d-acb8-3c4df76c27a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:26.632655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73957374-9ffe-4b8d-acb8-3c4df76c27a9 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:26.632932 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73957374-9ffe-4b8d-acb8-3c4df76c27a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:26.633204 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73957374-9ffe-4b8d-acb8-3c4df76c27a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:26.647188 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73957374-9ffe-4b8d-acb8-3c4df76c27a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:26.648637 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73957374-9ffe-4b8d-acb8-3c4df76c27a9 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:26.667291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-73957374-9ffe-4b8d-acb8-3c4df76c27a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:26.667291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:26.667291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:26.667291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:26.667291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:26.667291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:26.667291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:26.667291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:26.667291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:26.667291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:26.667291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:26.667291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:26.667291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:55:26.667291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:55:26.667291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:55:26.668165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:55:26.668165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:26.668165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:26.668165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:26.668165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:26.668165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:26.668165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:26.668165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:26.668165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:26.668165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:26.668165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:26.668165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:26.668165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:26.668165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:26.668165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:26.668165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:26.668165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:26.669094 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:55:26.669094 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:55:26.669094 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:26.669094 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:26.669094 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 487/985] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:26.675059 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03b215-a5ca-4533-8acb-33c88f95b5e6 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:26.776233 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03b215-a5ca-4533-8acb-33c88f95b5e6 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:26.811282 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4e1a2-770c-46b2-805c-fc9ce6c53542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:26.811901 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4e1a2-770c-46b2-805c-fc9ce6c53542 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:26.812179 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4e1a2-770c-46b2-805c-fc9ce6c53542 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:26.812449 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4e1a2-770c-46b2-805c-fc9ce6c53542 None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:26.813132 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba4e1a2-770c-46b2-805c-fc9ce6c53542 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:26.813434 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba4e1a2-770c-46b2-805c-fc9ce6c53542 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:26.814799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba4e1a2-770c-46b2-805c-fc9ce6c53542 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:26.821538 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:26.832226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 499/986] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:55:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:26.838649 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73957374-9ffe-4b8d-acb8-3c4df76c27a9 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:26.920772 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:26.941054 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73957374-9ffe-4b8d-acb8-3c4df76c27a9 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:26.976217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c25d3-659c-42fb-a5c6-8e70828c5809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:26.976862 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c25d3-659c-42fb-a5c6-8e70828c5809 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:26.977146 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c25d3-659c-42fb-a5c6-8e70828c5809 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:26.977423 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c25d3-659c-42fb-a5c6-8e70828c5809 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:26.984318 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889c25d3-659c-42fb-a5c6-8e70828c5809 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:26.985766 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889c25d3-659c-42fb-a5c6-8e70828c5809 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:26.998191 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-889c25d3-659c-42fb-a5c6-8e70828c5809 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:55:26.998191 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:26.998191 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:26.998191 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:26.998191 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:26.998191 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:26.998191 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:26.998191 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:26.998191 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:26.998191 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:26.998191 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:26.998191 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:26.998191 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:55:26.998191 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:55:26.998191 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:55:26.999403 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:55:26.999403 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:26.999403 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:26.999403 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:26.999403 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:26.999403 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:26.999403 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:26.999403 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:26.999403 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:26.999403 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:26.999403 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:26.999403 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:26.999403 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:26.999403 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:26.999403 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:26.999403 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:26.999403 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:27.000331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:55:27.000331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:55:27.000331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:55:27.000331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:55:27.000331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:55:27.000331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:55:27.000331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:55:27.000331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:27.003009 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 488/987] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:55:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:27.005487 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4e1a2-770c-46b2-805c-fc9ce6c53542 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:27.032105 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59228f4c-a246-4073-861e-c950df4bcd92 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:27.033105 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59228f4c-a246-4073-861e-c950df4bcd92 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:27.033877 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59228f4c-a246-4073-861e-c950df4bcd92 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:27.036261 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 489/988] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:55:27.041359 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04a9d2-739e-438c-8a18-e1067100b40f None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:27.042110 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04a9d2-739e-438c-8a18-e1067100b40f None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:27.043536 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04a9d2-739e-438c-8a18-e1067100b40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:27.067446 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-ba04a9d2-739e-438c-8a18-e1067100b40f None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:27.069669 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ba04a9d2-739e-438c-8a18-e1067100b40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:27.113822 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4e1a2-770c-46b2-805c-fc9ce6c53542 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:27.148596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce57897-b101-49d9-b894-d532d85cdd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:27.149215 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce57897-b101-49d9-b894-d532d85cdd63 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:27.149498 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce57897-b101-49d9-b894-d532d85cdd63 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:27.149775 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce57897-b101-49d9-b894-d532d85cdd63 None admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:27.150522 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce57897-b101-49d9-b894-d532d85cdd63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:27.150833 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce57897-b101-49d9-b894-d532d85cdd63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:27.152327 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce57897-b101-49d9-b894-d532d85cdd63 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:27.154583 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 490/989] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:27 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:55:27.165265 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 500/990] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:55:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:27.171106 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba04a9d2-739e-438c-8a18-e1067100b40f None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:27.171743 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce57897-b101-49d9-b894-d532d85cdd63 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:27.231743 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:27.247798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:27.274346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce57897-b101-49d9-b894-d532d85cdd63 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:27.289570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba04a9d2-739e-438c-8a18-e1067100b40f None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:27.312810 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e76b2-b41e-4848-8a68-8a7e7bb69326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:27.313431 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e76b2-b41e-4848-8a68-8a7e7bb69326 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:27.313709 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e76b2-b41e-4848-8a68-8a7e7bb69326 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:27.313983 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e76b2-b41e-4848-8a68-8a7e7bb69326 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:27.314761 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3e76b2-b41e-4848-8a68-8a7e7bb69326 None admin] RBAC: Adding query filter params (role.id=729f6ba7aded468788a72fefb5763fc4, scope.project.id=cf3763a37383456184a9ca9351b4ce03, user.id=de853a2dfffb4f05b6f47df2280ae4e1) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:27.315098 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3e76b2-b41e-4848-8a68-8a7e7bb69326 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:27.316525 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3e76b2-b41e-4848-8a68-8a7e7bb69326 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:27.322348 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 501/991] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:55:27 2022] GET /identity/v3/role_assignments?user.id=de853a2dfffb4f05b6f47df2280ae4e1&scope.project.id=cf3763a37383456184a9ca9351b4ce03&role.id=729f6ba7aded468788a72fefb5763fc4 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:27.323618 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b96f28-1d5d-492d-af0b-274fb60d66e8 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:27.323905 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b96f28-1d5d-492d-af0b-274fb60d66e8 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:27.324183 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b96f28-1d5d-492d-af0b-274fb60d66e8 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:27.325966 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 502/992] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:27.331538 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4a8e4-e150-48e5-ba22-d99d6263cb92 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:27.332535 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4a8e4-e150-48e5-ba22-d99d6263cb92 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:27.332535 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4a8e4-e150-48e5-ba22-d99d6263cb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:27.336849 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4bbdde-ddf3-4171-87dc-847547cae495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:27.337588 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4bbdde-ddf3-4171-87dc-847547cae495 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:27.337929 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4bbdde-ddf3-4171-87dc-847547cae495 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:27.338215 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4bbdde-ddf3-4171-87dc-847547cae495 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:27.339084 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4bbdde-ddf3-4171-87dc-847547cae495 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:27.341173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4bbdde-ddf3-4171-87dc-847547cae495 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:27.351425 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-8e4bbdde-ddf3-4171-87dc-847547cae495 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 04:55:27.351425 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:27.351425 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 04:55:27.351425 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 04:55:27.351425 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 04:55:27.351425 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 04:55:27.351425 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 04:55:27.351425 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:27.351425 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 04:55:27.351425 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:27.351425 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:55:27.351425 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:27.351425 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:27.351425 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:27.351425 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:27.351425 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:27.351425 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:27.352548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:27.352548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:27.352548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:27.352548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:27.352548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:27.352548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:27.352548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 04:55:27.352548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 04:55:27.352548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 04:55:27.352548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 04:55:27.352548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:27.352548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:27.352548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:27.352548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:27.352548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:27.352548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:27.368753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:27.368753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:27.368753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:27.368753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:27.368753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:27.368753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:27.368753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:27.368753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:27.368753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:27.368753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:27.368753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 04:55:27.368753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:27.368753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 04:55:27.368753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:27.368753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 491/993] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:55:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:27.369581 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-ccb4a8e4-e150-48e5-ba22-d99d6263cb92 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:27.369658 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4bbdde-ddf3-4171-87dc-847547cae495 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:27.369741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-ccb4a8e4-e150-48e5-ba22-d99d6263cb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:27.448103 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 503/994] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:27 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 04:55:27.470762 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4a8e4-e150-48e5-ba22-d99d6263cb92 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:27.474548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4bbdde-ddf3-4171-87dc-847547cae495 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:27.514171 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f098c8-364f-4102-9a55-6e3fe88ee526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:27.514948 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f098c8-364f-4102-9a55-6e3fe88ee526 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:27.515232 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f098c8-364f-4102-9a55-6e3fe88ee526 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:27.515503 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f098c8-364f-4102-9a55-6e3fe88ee526 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:27.516182 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f098c8-364f-4102-9a55-6e3fe88ee526 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:27.516487 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f098c8-364f-4102-9a55-6e3fe88ee526 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:27.517855 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f098c8-364f-4102-9a55-6e3fe88ee526 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:27.519127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:27.525281 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 492/995] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Aug 3 04:55:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:27.531767 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f098c8-364f-4102-9a55-6e3fe88ee526 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:27.542813 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:27.597616 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4a8e4-e150-48e5-ba22-d99d6263cb92 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:27.638601 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587fcbe-8ac6-42ce-959d-c04477a879f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:27.639295 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587fcbe-8ac6-42ce-959d-c04477a879f2 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:27.639579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587fcbe-8ac6-42ce-959d-c04477a879f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:27.639857 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587fcbe-8ac6-42ce-959d-c04477a879f2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:27.640498 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1587fcbe-8ac6-42ce-959d-c04477a879f2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:27.642024 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1587fcbe-8ac6-42ce-959d-c04477a879f2 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:27.646027 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f098c8-364f-4102-9a55-6e3fe88ee526 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:27.655034 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-1587fcbe-8ac6-42ce-959d-c04477a879f2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 04:55:27.655034 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:27.655034 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 04:55:27.655034 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 04:55:27.655034 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 04:55:27.655034 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 04:55:27.655034 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 04:55:27.655034 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:27.655034 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 04:55:27.655034 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:27.655034 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:55:27.655034 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:27.655034 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:27.655034 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:27.655034 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:27.655034 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:27.655034 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:27.656554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:27.656554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:27.656554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:27.656554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:27.656554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:27.656554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:27.656554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 04:55:27.656554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 04:55:27.656554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 04:55:27.656554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 04:55:27.656554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:27.656554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:27.656554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:27.656554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:27.656554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:27.656554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:27.657960 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:27.657960 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:27.657960 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:27.657960 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:27.657960 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:27.657960 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:27.657960 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:27.657960 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:27.657960 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:27.657960 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:27.657960 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 04:55:27.657960 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:27.657960 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 04:55:27.657960 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:27.666167 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 504/996] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Aug 3 04:55:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:55:27.667115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587fcbe-8ac6-42ce-959d-c04477a879f2 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:27.691504 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d20626-9516-474e-99a2-0be93719ab4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:27.692397 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d20626-9516-474e-99a2-0be93719ab4f None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:27.692680 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d20626-9516-474e-99a2-0be93719ab4f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:27.692955 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d20626-9516-474e-99a2-0be93719ab4f None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:27.693745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d20626-9516-474e-99a2-0be93719ab4f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:27.694047 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d20626-9516-474e-99a2-0be93719ab4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:27.695437 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d20626-9516-474e-99a2-0be93719ab4f None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:27.705639 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 493/997] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Aug 3 04:55:27 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:27.707236 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d20626-9516-474e-99a2-0be93719ab4f None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:27.772498 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:27.774434 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587fcbe-8ac6-42ce-959d-c04477a879f2 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:27.791627 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:27.811547 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed59dc-d0ed-420b-b1a2-0c9097d0dc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:27.812382 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed59dc-d0ed-420b-b1a2-0c9097d0dc54 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:27.812818 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed59dc-d0ed-420b-b1a2-0c9097d0dc54 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:27.813361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed59dc-d0ed-420b-b1a2-0c9097d0dc54 None admin] PATH_INFO: `/v3/services` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:27.814064 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed59dc-d0ed-420b-b1a2-0c9097d0dc54 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:27.814376 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed59dc-d0ed-420b-b1a2-0c9097d0dc54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:27.815685 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d20626-9516-474e-99a2-0be93719ab4f None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:27.815807 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed59dc-d0ed-420b-b1a2-0c9097d0dc54 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:27.821040 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 505/998] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Aug 3 04:55:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:27.826513 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed59dc-d0ed-420b-b1a2-0c9097d0dc54 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:27.852198 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1e9e8-aa45-4019-9032-e5560b5f791d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:27.852933 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1e9e8-aa45-4019-9032-e5560b5f791d None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:27.853095 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1e9e8-aa45-4019-9032-e5560b5f791d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:27.853399 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1e9e8-aa45-4019-9032-e5560b5f791d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:27.854188 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e1e9e8-aa45-4019-9032-e5560b5f791d None admin] RBAC: Adding query filter params (interface=public, service_id=ee0eb115e22c44c8b2f96935bc7bf721, region_id=RegionOne) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:27.854439 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e1e9e8-aa45-4019-9032-e5560b5f791d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:27.855897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e1e9e8-aa45-4019-9032-e5560b5f791d None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:27.866128 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 494/999] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Aug 3 04:55:27 2022] GET /identity/v3/endpoints?service_id=ee0eb115e22c44c8b2f96935bc7bf721&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:27.867430 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1e9e8-aa45-4019-9032-e5560b5f791d None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:27.936074 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed59dc-d0ed-420b-b1a2-0c9097d0dc54 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:27.974263 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1e9e8-aa45-4019-9032-e5560b5f791d None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:27.976250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b2280-dcae-4afd-bfe7-3cbee0f02cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:27.976887 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b2280-dcae-4afd-bfe7-3cbee0f02cab None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:27.977165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b2280-dcae-4afd-bfe7-3cbee0f02cab None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:27.977437 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b2280-dcae-4afd-bfe7-3cbee0f02cab None admin] PATH_INFO: `/v3/services` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:27.978112 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b2280-dcae-4afd-bfe7-3cbee0f02cab None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:27.978416 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b2280-dcae-4afd-bfe7-3cbee0f02cab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:27.979811 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b2280-dcae-4afd-bfe7-3cbee0f02cab None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:27.989443 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 506/1000] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Aug 3 04:55:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:27.990445 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b2280-dcae-4afd-bfe7-3cbee0f02cab None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:28.006999 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:28.017176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630ed61a-49a8-4fc2-b502-257e75202b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:28.017802 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ed61a-49a8-4fc2-b502-257e75202b4c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:28.018089 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ed61a-49a8-4fc2-b502-257e75202b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:28.018422 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ed61a-49a8-4fc2-b502-257e75202b4c None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:28.019097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630ed61a-49a8-4fc2-b502-257e75202b4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:28.025396 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630ed61a-49a8-4fc2-b502-257e75202b4c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:28.034880 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 495/1001] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Aug 3 04:55:27 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 04:55:28.048314 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:28.110536 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b2280-dcae-4afd-bfe7-3cbee0f02cab None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:28.147707 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9d398-d8ee-497a-bbc4-1f67eec9c94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:28.148349 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9d398-d8ee-497a-bbc4-1f67eec9c94d None admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:28.148628 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9d398-d8ee-497a-bbc4-1f67eec9c94d None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:28.148902 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9d398-d8ee-497a-bbc4-1f67eec9c94d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:28.149623 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9d398-d8ee-497a-bbc4-1f67eec9c94d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:28.151246 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9d398-d8ee-497a-bbc4-1f67eec9c94d None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:28.220304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 507/1002] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 04:55:27 2022] POST /identity/v3/endpoints => generated 325 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:55:29.376210 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b0e2a-d58f-4823-a953-4a4b0f021b02 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:29.376733 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b0e2a-d58f-4823-a953-4a4b0f021b02 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:29.376812 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b0e2a-d58f-4823-a953-4a4b0f021b02 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:29.378367 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 496/1003] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:29.383368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7e185-d992-4310-80d1-2d3d9e101e0f None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:29.383759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7e185-d992-4310-80d1-2d3d9e101e0f None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:29.383927 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7e185-d992-4310-80d1-2d3d9e101e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:29.402342 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-87a7e185-d992-4310-80d1-2d3d9e101e0f None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:29.404140 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-87a7e185-d992-4310-80d1-2d3d9e101e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:29.476876 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 508/1004] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:29 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:55:29.498691 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b0e2a-d58f-4823-a953-4a4b0f021b02 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:29.609474 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b0e2a-d58f-4823-a953-4a4b0f021b02 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:29.648372 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3723a95f-6ef9-4af7-973c-acf793795e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:29.648934 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723a95f-6ef9-4af7-973c-acf793795e36 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:29.649218 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723a95f-6ef9-4af7-973c-acf793795e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:29.649498 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723a95f-6ef9-4af7-973c-acf793795e36 None admin] PATH_INFO: `/v3/services/image` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:29.650130 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3723a95f-6ef9-4af7-973c-acf793795e36 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:29.651629 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3723a95f-6ef9-4af7-973c-acf793795e36 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:29.661447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-3723a95f-6ef9-4af7-973c-acf793795e36 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 04:55:29.661447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:29.661447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 04:55:29.661447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 04:55:29.661447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 04:55:29.661447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 04:55:29.661447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 04:55:29.661447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:29.661447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 04:55:29.661447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:29.661447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:55:29.661447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:29.661447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:29.661447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:29.661447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:29.661447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:29.661447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:29.662470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:29.662470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:29.662470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:29.662470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:29.662470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:29.662470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:29.662470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 04:55:29.662470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 04:55:29.662470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 04:55:29.662470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 04:55:29.662470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:29.662470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:29.662470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:29.662470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:29.662470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:29.662470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:29.663817 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:29.663817 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:29.663817 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:29.663817 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:29.663817 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:29.663817 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:29.663817 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:29.663817 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:29.663817 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:29.663817 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:29.663817 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 04:55:29.663817 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:29.663817 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 04:55:29.663817 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:29.663817 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 497/1005] 10.222.0.40 () {56 vars in 1105 bytes} [Wed Aug 3 04:55:29 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:29.671731 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a7e185-d992-4310-80d1-2d3d9e101e0f None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:29.703420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:29.776583 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a7e185-d992-4310-80d1-2d3d9e101e0f None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:29.812356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014ce93-3772-498e-bc4d-7f6d090c00d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:29.812983 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014ce93-3772-498e-bc4d-7f6d090c00d1 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:29.813269 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014ce93-3772-498e-bc4d-7f6d090c00d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:29.813544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014ce93-3772-498e-bc4d-7f6d090c00d1 None admin] PATH_INFO: `/v3/services` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:29.814224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014ce93-3772-498e-bc4d-7f6d090c00d1 None admin] RBAC: Adding query filter params (name=image) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:29.814533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014ce93-3772-498e-bc4d-7f6d090c00d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:29.815944 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014ce93-3772-498e-bc4d-7f6d090c00d1 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:29.821045 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 509/1006] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:29.827095 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3723a95f-6ef9-4af7-973c-acf793795e36 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:29.884794 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:29.932018 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3723a95f-6ef9-4af7-973c-acf793795e36 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:29.966863 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fea788-9bac-49bf-bf30-b73cc15ec8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:29.967496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fea788-9bac-49bf-bf30-b73cc15ec8ef None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:29.967774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fea788-9bac-49bf-bf30-b73cc15ec8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:29.968288 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fea788-9bac-49bf-bf30-b73cc15ec8ef None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:29.968716 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fea788-9bac-49bf-bf30-b73cc15ec8ef None admin] RBAC: Adding query filter params (type=image) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:29.969017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fea788-9bac-49bf-bf30-b73cc15ec8ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:29.970384 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fea788-9bac-49bf-bf30-b73cc15ec8ef None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:29.975534 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 498/1007] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:29 2022] GET /identity/v3/services?type=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:30.067982 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e263cf-8823-4c2f-a598-3649133767c8 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:30.068368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e263cf-8823-4c2f-a598-3649133767c8 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:30.068547 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e263cf-8823-4c2f-a598-3649133767c8 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:30.069930 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 510/1008] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:30.076197 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7014e6e-ac18-4a6b-8fe9-cc3a56ed785d None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:30.076492 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7014e6e-ac18-4a6b-8fe9-cc3a56ed785d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:30.076773 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7014e6e-ac18-4a6b-8fe9-cc3a56ed785d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:30.095848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-c7014e6e-ac18-4a6b-8fe9-cc3a56ed785d None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:30.097060 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c7014e6e-ac18-4a6b-8fe9-cc3a56ed785d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:30.171412 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 499/1009] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:30 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:55:30.194281 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e263cf-8823-4c2f-a598-3649133767c8 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:30.256600 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205fe52-5252-4d94-8845-eb3e32aadad3 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:30.256911 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205fe52-5252-4d94-8845-eb3e32aadad3 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:30.257172 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205fe52-5252-4d94-8845-eb3e32aadad3 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:30.258516 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 500/1010] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:55:30.263374 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a4d0d-8127-4aa3-b3c4-78ed28222035 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:30.263658 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a4d0d-8127-4aa3-b3c4-78ed28222035 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:30.263931 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a4d0d-8127-4aa3-b3c4-78ed28222035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:30.282530 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-7b2a4d0d-8127-4aa3-b3c4-78ed28222035 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:30.283762 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7b2a4d0d-8127-4aa3-b3c4-78ed28222035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:30.294329 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:30.302404 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e263cf-8823-4c2f-a598-3649133767c8 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:30.339226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a794f7f-a23c-4035-8b2f-d2cbf97f9c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:30.339889 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a794f7f-a23c-4035-8b2f-d2cbf97f9c3e None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:30.340234 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a794f7f-a23c-4035-8b2f-d2cbf97f9c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:30.340586 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a794f7f-a23c-4035-8b2f-d2cbf97f9c3e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:30.341435 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a794f7f-a23c-4035-8b2f-d2cbf97f9c3e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:30.343801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a794f7f-a23c-4035-8b2f-d2cbf97f9c3e None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:30.352254 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:30.358304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-6a794f7f-a23c-4035-8b2f-d2cbf97f9c3e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 04:55:30.358304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:30.358304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 04:55:30.358304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 04:55:30.358304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 04:55:30.358304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 04:55:30.358304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 04:55:30.358304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:30.358304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 04:55:30.358304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:30.358304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:55:30.358304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:30.358304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:30.358304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:30.358304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:30.358304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:30.358304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:30.359359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:30.359359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:30.359359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:30.359359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:30.359359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:30.359359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:30.359359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 04:55:30.359359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 04:55:30.359359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 04:55:30.359359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 04:55:30.359359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:30.359359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:30.359359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:30.359359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:30.359359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:30.359359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:30.360250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:30.360250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:30.360250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:30.360250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:30.360250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:30.360250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:30.360250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:30.360250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:30.360250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:30.360250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:30.360250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 04:55:30.360250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:30.360250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 04:55:30.360250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:30.360250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 511/1011] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:55:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:30.366139 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a794f7f-a23c-4035-8b2f-d2cbf97f9c3e None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:30.380964 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 501/1012] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:30 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 04:55:30.404457 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a4d0d-8127-4aa3-b3c4-78ed28222035 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:30.487299 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a794f7f-a23c-4035-8b2f-d2cbf97f9c3e None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:30.520239 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a4d0d-8127-4aa3-b3c4-78ed28222035 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:30.527241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3962c8-0f90-494e-ac6e-255f6b3481f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:30.528373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3962c8-0f90-494e-ac6e-255f6b3481f9 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:30.529085 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3962c8-0f90-494e-ac6e-255f6b3481f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:30.529401 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3962c8-0f90-494e-ac6e-255f6b3481f9 None admin] PATH_INFO: `/v3/services` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:30.530085 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3962c8-0f90-494e-ac6e-255f6b3481f9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:30.530391 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3962c8-0f90-494e-ac6e-255f6b3481f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:30.532553 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3962c8-0f90-494e-ac6e-255f6b3481f9 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:30.542871 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 512/1013] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Aug 3 04:55:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:30.544268 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3962c8-0f90-494e-ac6e-255f6b3481f9 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:30.558695 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04091b24-345a-4416-a426-137c6620df72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:30.559506 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04091b24-345a-4416-a426-137c6620df72 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:30.559856 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04091b24-345a-4416-a426-137c6620df72 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:30.560140 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04091b24-345a-4416-a426-137c6620df72 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:30.561314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04091b24-345a-4416-a426-137c6620df72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:30.562773 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04091b24-345a-4416-a426-137c6620df72 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:30.566884 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 502/1014] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:30.570025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04091b24-345a-4416-a426-137c6620df72 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:30.575541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:30.622584 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:30.662631 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3962c8-0f90-494e-ac6e-255f6b3481f9 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:30.684421 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04091b24-345a-4416-a426-137c6620df72 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:30.699699 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c10bba-9fd9-4957-850c-fbe14adc443d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:30.700322 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c10bba-9fd9-4957-850c-fbe14adc443d None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:30.700603 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c10bba-9fd9-4957-850c-fbe14adc443d None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:30.700874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c10bba-9fd9-4957-850c-fbe14adc443d None admin] PATH_INFO: `/v3/services` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:30.701542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c10bba-9fd9-4957-850c-fbe14adc443d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:30.701848 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c10bba-9fd9-4957-850c-fbe14adc443d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:30.703245 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c10bba-9fd9-4957-850c-fbe14adc443d None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:30.708141 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 513/1015] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Aug 3 04:55:30 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:30.713755 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c10bba-9fd9-4957-850c-fbe14adc443d None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:30.729250 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121bf28-f0f2-4bf1-a312-da5e51b20409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:30.730121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121bf28-f0f2-4bf1-a312-da5e51b20409 None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:30.730121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121bf28-f0f2-4bf1-a312-da5e51b20409 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:30.730439 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121bf28-f0f2-4bf1-a312-da5e51b20409 None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:30.731682 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d121bf28-f0f2-4bf1-a312-da5e51b20409 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:30.733151 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d121bf28-f0f2-4bf1-a312-da5e51b20409 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:30.801370 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:30.816363 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c10bba-9fd9-4957-850c-fbe14adc443d None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:30.821461 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 503/1016] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Aug 3 04:55:30 2022] POST /identity/v3/projects => generated 311 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 04:55:30.854409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74794a2e-eaf8-461a-92ea-684dd0563158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:30.855071 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74794a2e-eaf8-461a-92ea-684dd0563158 None admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:30.855412 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74794a2e-eaf8-461a-92ea-684dd0563158 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:30.856262 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74794a2e-eaf8-461a-92ea-684dd0563158 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:30.857109 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74794a2e-eaf8-461a-92ea-684dd0563158 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:30.858518 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74794a2e-eaf8-461a-92ea-684dd0563158 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:30.958865 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 514/1017] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 04:55:30 2022] POST /identity/v3/endpoints => generated 334 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 04:55:31.999236 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b1b7f-315c-46d7-a383-1234e08a8aa8 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:31.999758 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b1b7f-315c-46d7-a383-1234e08a8aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:31.999849 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b1b7f-315c-46d7-a383-1234e08a8aa8 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:32.001315 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 504/1018] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:32.006238 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab49b6-be75-4594-880c-f865cc9c7fcd None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:32.006644 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab49b6-be75-4594-880c-f865cc9c7fcd None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:32.006830 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab49b6-be75-4594-880c-f865cc9c7fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:32.025170 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-b2ab49b6-be75-4594-880c-f865cc9c7fcd None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:32.026362 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-b2ab49b6-be75-4594-880c-f865cc9c7fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:32.101453 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 515/1019] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:32 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:55:32.123773 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b1b7f-315c-46d7-a383-1234e08a8aa8 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:32.227993 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:32.233271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b1b7f-315c-46d7-a383-1234e08a8aa8 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:32.267666 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423c563-8f32-4a9a-b546-95e318dbe8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:32.268207 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423c563-8f32-4a9a-b546-95e318dbe8f8 None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:32.268483 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423c563-8f32-4a9a-b546-95e318dbe8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:32.268754 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423c563-8f32-4a9a-b546-95e318dbe8f8 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:32.269482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e423c563-8f32-4a9a-b546-95e318dbe8f8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:32.270857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e423c563-8f32-4a9a-b546-95e318dbe8f8 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:32.350892 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 505/1020] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Aug 3 04:55:32 2022] POST /identity/v3/services => generated 242 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:55:32.882284 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a39fc7-609f-4366-a651-368d3bd4c390 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:32.882607 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a39fc7-609f-4366-a651-368d3bd4c390 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:32.882856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a39fc7-609f-4366-a651-368d3bd4c390 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:32.884251 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 516/1021] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:32.889887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc2dc6-76b3-4b07-b107-577e74430e68 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:32.890179 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc2dc6-76b3-4b07-b107-577e74430e68 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:32.890456 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc2dc6-76b3-4b07-b107-577e74430e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:32.908721 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-09bc2dc6-76b3-4b07-b107-577e74430e68 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:32.909825 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-09bc2dc6-76b3-4b07-b107-577e74430e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:32.983182 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 506/1022] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:32 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 04:55:33.007634 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a39fc7-609f-4366-a651-368d3bd4c390 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:33.085447 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:33.108102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fcf434-3c17-45ed-852d-8569130b2d5e None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:33.108617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fcf434-3c17-45ed-852d-8569130b2d5e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:33.108709 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fcf434-3c17-45ed-852d-8569130b2d5e None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:33.110133 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 507/1023] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:55:33.114913 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3af17f-8a8f-4ea3-a7cf-29f325524cbf None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:33.115202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3af17f-8a8f-4ea3-a7cf-29f325524cbf None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:33.115482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3af17f-8a8f-4ea3-a7cf-29f325524cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:33.120908 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a39fc7-609f-4366-a651-368d3bd4c390 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:33.134122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-9a3af17f-8a8f-4ea3-a7cf-29f325524cbf None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:33.135346 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9a3af17f-8a8f-4ea3-a7cf-29f325524cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:33.161754 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4546356-3f29-453a-aaea-381974f08fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:33.162954 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4546356-3f29-453a-aaea-381974f08fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:33.163397 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4546356-3f29-453a-aaea-381974f08fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:33.163823 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4546356-3f29-453a-aaea-381974f08fa1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:33.165148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4546356-3f29-453a-aaea-381974f08fa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:33.166582 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4546356-3f29-453a-aaea-381974f08fa1 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:33.168737 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 517/1024] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:33.174498 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4546356-3f29-453a-aaea-381974f08fa1 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:33.213572 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 508/1025] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:33 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 04:55:33.234877 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3af17f-8a8f-4ea3-a7cf-29f325524cbf None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:33.251144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:33.281739 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4546356-3f29-453a-aaea-381974f08fa1 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:33.322611 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3499ae1-2b1e-4292-a3d7-24fc93049e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:33.323603 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3499ae1-2b1e-4292-a3d7-24fc93049e03 None admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:33.323886 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3499ae1-2b1e-4292-a3d7-24fc93049e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:33.324159 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3499ae1-2b1e-4292-a3d7-24fc93049e03 None admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:33.324861 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3499ae1-2b1e-4292-a3d7-24fc93049e03 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:33.326904 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3499ae1-2b1e-4292-a3d7-24fc93049e03 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:33.338465 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:33.343108 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3af17f-8a8f-4ea3-a7cf-29f325524cbf None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:33.378055 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41fbcf6-bf2c-4d75-ba90-9a2f083f84a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:33.378709 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fbcf6-bf2c-4d75-ba90-9a2f083f84a8 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:33.379008 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fbcf6-bf2c-4d75-ba90-9a2f083f84a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:33.379289 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fbcf6-bf2c-4d75-ba90-9a2f083f84a8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:33.389605 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41fbcf6-bf2c-4d75-ba90-9a2f083f84a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:55:33.394482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-a41fbcf6-bf2c-4d75-ba90-9a2f083f84a8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 04:55:33.394482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:33.394482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:33.394482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:33.394482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:33.394482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:33.394482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:33.394482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:33.394482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:33.394482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:33.394482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:33.394482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:33.394482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:55:33.394482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:55:33.394482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:55:33.394482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:55:33.395656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:55:33.395656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:55:33.395656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:55:33.395656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:55:33.395656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:33.395656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:33.395656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:33.395656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:33.395656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:33.395656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:33.395656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:33.395656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:33.395656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:33.395656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:33.395656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:33.395656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:33.396830 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:33.396830 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:33.396830 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:33.396830 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:33.396830 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:55:33.396830 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:55:33.396830 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:55:33.396830 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:55:33.396830 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:55:33.396830 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:55:33.396830 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 04:55:33.396830 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:33.396830 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 509/1026] 10.222.0.40 () {56 vars in 1120 bytes} [Wed Aug 3 04:55:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:55:33.402082 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41fbcf6-bf2c-4d75-ba90-9a2f083f84a8 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:33.414712 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 518/1027] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 3 04:55:33 2022] POST /identity/v3/users => generated 285 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 04:55:33.504089 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:33.532249 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41fbcf6-bf2c-4d75-ba90-9a2f083f84a8 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:33.572175 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d6dfb-cc44-42a7-8b85-2694b3ad573a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:33.572796 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d6dfb-cc44-42a7-8b85-2694b3ad573a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:33.573826 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d6dfb-cc44-42a7-8b85-2694b3ad573a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:33.573826 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d6dfb-cc44-42a7-8b85-2694b3ad573a None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:33.574060 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9d6dfb-cc44-42a7-8b85-2694b3ad573a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:33.574395 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9d6dfb-cc44-42a7-8b85-2694b3ad573a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:33.575852 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9d6dfb-cc44-42a7-8b85-2694b3ad573a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:33.585530 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 510/1028] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Aug 3 04:55:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:33.592002 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3499ae1-2b1e-4292-a3d7-24fc93049e03 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:33.701134 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3499ae1-2b1e-4292-a3d7-24fc93049e03 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:33.736267 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c779e0-1363-4d54-8aed-db5544c4bb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:33.737015 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c779e0-1363-4d54-8aed-db5544c4bb1d None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:33.737303 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c779e0-1363-4d54-8aed-db5544c4bb1d None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:33.737584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c779e0-1363-4d54-8aed-db5544c4bb1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:33.748497 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c779e0-1363-4d54-8aed-db5544c4bb1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:33.749961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c779e0-1363-4d54-8aed-db5544c4bb1d None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:33.764624 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-66c779e0-1363-4d54-8aed-db5544c4bb1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:33.764624 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:33.764624 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:33.764624 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:33.764624 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:33.764624 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:33.764624 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:33.764624 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:33.764624 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:33.764624 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:33.764624 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:33.764624 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:33.764624 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:55:33.764624 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:55:33.764624 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:55:33.765540 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:55:33.765540 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:33.765540 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:33.765540 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:33.765540 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:33.765540 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:33.765540 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:33.765540 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:33.765540 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:33.765540 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:33.765540 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:33.765540 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:33.765540 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:33.765540 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:33.765540 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:33.765540 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:33.765540 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:33.766472 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:55:33.766472 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:55:33.766472 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:33.766472 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:33.766472 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 519/1029] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:33.773042 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d6dfb-cc44-42a7-8b85-2694b3ad573a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:33.877404 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d6dfb-cc44-42a7-8b85-2694b3ad573a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:33.911973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734c240-19bb-4982-900c-7cf64db20875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:33.912600 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734c240-19bb-4982-900c-7cf64db20875 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:33.912885 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734c240-19bb-4982-900c-7cf64db20875 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:33.913165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734c240-19bb-4982-900c-7cf64db20875 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:33.913859 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4734c240-19bb-4982-900c-7cf64db20875 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:33.914169 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4734c240-19bb-4982-900c-7cf64db20875 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:33.915589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4734c240-19bb-4982-900c-7cf64db20875 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:33.918863 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:33.934036 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 511/1030] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:33.935446 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c779e0-1363-4d54-8aed-db5544c4bb1d None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:33.985185 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:34.040517 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c779e0-1363-4d54-8aed-db5544c4bb1d None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:34.075352 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0508d-61ce-42d8-b1fe-314d65694f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:34.075989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0508d-61ce-42d8-b1fe-314d65694f1c None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:34.076269 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0508d-61ce-42d8-b1fe-314d65694f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:34.076540 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0508d-61ce-42d8-b1fe-314d65694f1c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:34.084324 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a0508d-61ce-42d8-b1fe-314d65694f1c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:34.085776 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a0508d-61ce-42d8-b1fe-314d65694f1c None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:34.097742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-a3a0508d-61ce-42d8-b1fe-314d65694f1c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 04:55:34.097742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:34.097742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:34.097742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:34.097742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:34.097742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:34.097742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:34.097742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:34.097742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:34.097742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:34.097742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:34.097742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:34.097742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:55:34.097742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:55:34.097742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:55:34.097742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:55:34.098732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:34.098732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:34.098732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:55:34.098732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:55:34.098732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:55:34.098732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:55:34.098732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:34.098732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:34.098732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:34.098732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:34.098732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:34.098732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:34.098732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:34.098732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:34.098732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:34.098732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:34.098732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:34.098732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:34.104549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:34.104549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:34.104549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:55:34.104549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:55:34.104549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:55:34.104549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:55:34.104549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:55:34.104549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:55:34.104549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 04:55:34.104549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:34.104549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 520/1031] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Aug 3 04:55:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:34.105185 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734c240-19bb-4982-900c-7cf64db20875 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:34.205206 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734c240-19bb-4982-900c-7cf64db20875 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:34.239233 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200856d-ee8e-49f8-a636-f0e6e464f95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:34.239846 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200856d-ee8e-49f8-a636-f0e6e464f95d None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:34.240127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200856d-ee8e-49f8-a636-f0e6e464f95d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:34.240399 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200856d-ee8e-49f8-a636-f0e6e464f95d None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:34.241182 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b200856d-ee8e-49f8-a636-f0e6e464f95d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:34.241489 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b200856d-ee8e-49f8-a636-f0e6e464f95d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:34.242928 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b200856d-ee8e-49f8-a636-f0e6e464f95d None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:34.265285 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 512/1032] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Aug 3 04:55:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:34.266479 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0508d-61ce-42d8-b1fe-314d65694f1c None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:34.368894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0508d-61ce-42d8-b1fe-314d65694f1c None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:34.371730 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:34.404845 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28233d1d-dd4e-4a5f-b579-53ac742a0561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:34.405462 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28233d1d-dd4e-4a5f-b579-53ac742a0561 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:34.405742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28233d1d-dd4e-4a5f-b579-53ac742a0561 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:34.405975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b17b0-c09b-4f0a-8991-2f1a0bcf7565 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:34.406080 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28233d1d-dd4e-4a5f-b579-53ac742a0561 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:34.406272 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b17b0-c09b-4f0a-8991-2f1a0bcf7565 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:34.406556 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b17b0-c09b-4f0a-8991-2f1a0bcf7565 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:34.408086 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 513/1033] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:55:34.412795 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2226caf-8c98-43d6-af58-c47bcfcf63fd None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:34.413086 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2226caf-8c98-43d6-af58-c47bcfcf63fd None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:34.413364 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2226caf-8c98-43d6-af58-c47bcfcf63fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:34.431037 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-b2226caf-8c98-43d6-af58-c47bcfcf63fd None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:34.432233 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b2226caf-8c98-43d6-af58-c47bcfcf63fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:34.484950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28233d1d-dd4e-4a5f-b579-53ac742a0561 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:34.486655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28233d1d-dd4e-4a5f-b579-53ac742a0561 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:34.491603 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:34.512727 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 514/1034] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:34 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 04:55:34.535504 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2226caf-8c98-43d6-af58-c47bcfcf63fd None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:34.602885 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 521/1035] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:55:34 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 03 04:55:34.604537 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28233d1d-dd4e-4a5f-b579-53ac742a0561 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:34.608208 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:34.650556 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2226caf-8c98-43d6-af58-c47bcfcf63fd None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:34.690686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d218d4-31ef-405c-97a8-57dc02a63b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:34.691359 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d218d4-31ef-405c-97a8-57dc02a63b16 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:34.691641 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d218d4-31ef-405c-97a8-57dc02a63b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:34.691959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d218d4-31ef-405c-97a8-57dc02a63b16 None admin] PATH_INFO: `/v3/services/image` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:34.693055 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d218d4-31ef-405c-97a8-57dc02a63b16 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:34.695596 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d218d4-31ef-405c-97a8-57dc02a63b16 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:34.712218 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-77d218d4-31ef-405c-97a8-57dc02a63b16 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 04:55:34.712218 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:34.712218 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 04:55:34.712218 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 04:55:34.712218 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 04:55:34.712218 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 04:55:34.712218 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 04:55:34.712218 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:34.712218 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 04:55:34.712218 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:34.712218 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:55:34.712218 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:34.712218 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:34.712218 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:34.712218 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:34.712218 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:34.712218 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:34.714032 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:34.714032 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:34.714032 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:34.714032 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:34.714032 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:34.714032 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:34.714032 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 04:55:34.714032 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 04:55:34.714032 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 04:55:34.714032 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 04:55:34.714032 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:34.714032 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:34.714032 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:34.714032 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:34.714032 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:34.714032 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:34.715001 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:34.715001 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:34.715001 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:34.715001 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:34.715001 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:34.715001 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:34.715001 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:34.715001 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:34.715001 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:34.715001 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:34.715001 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 04:55:34.715001 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:34.715001 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 04:55:34.715001 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:34.715001 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 515/1036] 10.222.0.40 () {56 vars in 1105 bytes} [Wed Aug 3 04:55:34 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:55:34.719491 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d218d4-31ef-405c-97a8-57dc02a63b16 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:34.740344 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28233d1d-dd4e-4a5f-b579-53ac742a0561 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:34.781606 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e862a92-1e67-4654-97a0-067a520c4407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:34.782257 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e862a92-1e67-4654-97a0-067a520c4407 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:34.782542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e862a92-1e67-4654-97a0-067a520c4407 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:34.782815 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e862a92-1e67-4654-97a0-067a520c4407 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:34.789381 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:34.797044 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e862a92-1e67-4654-97a0-067a520c4407 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:34.798517 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e862a92-1e67-4654-97a0-067a520c4407 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:34.814050 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-9e862a92-1e67-4654-97a0-067a520c4407 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:34.814050 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:34.814050 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:34.814050 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:34.814050 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:34.814050 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:34.814050 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:34.814050 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:34.814050 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:34.814050 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:34.814050 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:34.814050 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:34.814050 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:55:34.814050 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:55:34.814050 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:55:34.815235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:55:34.815235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:34.815235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:34.815235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:34.815235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:34.815235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:34.815235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:34.815235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:34.815235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:34.815235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:34.815235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:34.815235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:34.815235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:34.815235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:34.815235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:34.815235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:34.815235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:34.816338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:55:34.816338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:55:34.816338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:34.816338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:34.817113 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 522/1037] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:55:34.822542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e862a92-1e67-4654-97a0-067a520c4407 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:34.847246 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d218d4-31ef-405c-97a8-57dc02a63b16 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:34.882998 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:34.884302 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccbdfb1-9e49-443a-a2d6-54ae20e569d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:34.884916 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccbdfb1-9e49-443a-a2d6-54ae20e569d5 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:34.885199 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccbdfb1-9e49-443a-a2d6-54ae20e569d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:34.885475 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccbdfb1-9e49-443a-a2d6-54ae20e569d5 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:34.886147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccbdfb1-9e49-443a-a2d6-54ae20e569d5 None admin] RBAC: Adding query filter params (name=image) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:34.886450 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccbdfb1-9e49-443a-a2d6-54ae20e569d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:34.887981 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccbdfb1-9e49-443a-a2d6-54ae20e569d5 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:34.898011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 516/1038] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:34.899643 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccbdfb1-9e49-443a-a2d6-54ae20e569d5 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:34.933082 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e862a92-1e67-4654-97a0-067a520c4407 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:34.968225 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146da63d-3a16-431a-8c74-2554a3650398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:34.968842 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146da63d-3a16-431a-8c74-2554a3650398 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:34.969120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146da63d-3a16-431a-8c74-2554a3650398 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:34.969390 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146da63d-3a16-431a-8c74-2554a3650398 None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:34.970072 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146da63d-3a16-431a-8c74-2554a3650398 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:34.970379 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146da63d-3a16-431a-8c74-2554a3650398 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:34.971911 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146da63d-3a16-431a-8c74-2554a3650398 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:34.984471 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 523/1039] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:55:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:34.989932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146da63d-3a16-431a-8c74-2554a3650398 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:35.017992 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccbdfb1-9e49-443a-a2d6-54ae20e569d5 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:35.035769 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:35.059492 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a8fe55-5824-44d4-b78e-f3e86159aaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:35.060141 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8fe55-5824-44d4-b78e-f3e86159aaa8 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:35.060425 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8fe55-5824-44d4-b78e-f3e86159aaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:35.060702 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8fe55-5824-44d4-b78e-f3e86159aaa8 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:35.061398 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a8fe55-5824-44d4-b78e-f3e86159aaa8 None admin] RBAC: Adding query filter params (type=image) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:35.061706 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a8fe55-5824-44d4-b78e-f3e86159aaa8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:35.063446 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a8fe55-5824-44d4-b78e-f3e86159aaa8 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:35.075691 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 517/1040] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:34 2022] GET /identity/v3/services?type=image => generated 350 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:35.076622 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a8fe55-5824-44d4-b78e-f3e86159aaa8 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:35.130741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146da63d-3a16-431a-8c74-2554a3650398 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:35.154334 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:35.176419 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471e393-92ab-4025-9310-be62808a2581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:35.177289 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471e393-92ab-4025-9310-be62808a2581 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:35.177697 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471e393-92ab-4025-9310-be62808a2581 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:35.178168 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471e393-92ab-4025-9310-be62808a2581 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:35.187156 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471e393-92ab-4025-9310-be62808a2581 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:35.189046 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471e393-92ab-4025-9310-be62808a2581 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:35.201494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-3471e393-92ab-4025-9310-be62808a2581 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:55:35.201494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:35.201494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:35.201494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:35.201494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:35.201494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:35.201494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:35.201494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:35.201494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:35.201494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:35.201494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:35.201494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:35.201494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:55:35.201494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:55:35.201494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:55:35.202432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:55:35.202432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:35.202432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:35.202432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:35.202432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:35.202432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:35.202432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:35.202432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:35.202432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:35.202432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:35.202432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:35.202432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:35.202432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:35.202432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:35.202432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:35.202432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:35.202432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:35.203385 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:55:35.203385 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:55:35.203385 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:55:35.203385 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:55:35.203385 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:55:35.203385 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:55:35.203385 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:55:35.203385 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:35.203385 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 524/1041] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:55:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:35.208299 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471e393-92ab-4025-9310-be62808a2581 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:35.211273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a8fe55-5824-44d4-b78e-f3e86159aaa8 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:35.248638 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7919699c-1454-4136-9752-09fb6bdb85a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:35.249310 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919699c-1454-4136-9752-09fb6bdb85a0 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:35.249595 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919699c-1454-4136-9752-09fb6bdb85a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:35.249866 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919699c-1454-4136-9752-09fb6bdb85a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:35.250624 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7919699c-1454-4136-9752-09fb6bdb85a0 None admin] RBAC: Adding query filter params (interface=public, service_id=3adf4162784c42dea7f1005f9954b7b5, region_id=RegionOne) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:35.250965 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7919699c-1454-4136-9752-09fb6bdb85a0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:35.252338 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7919699c-1454-4136-9752-09fb6bdb85a0 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:35.257692 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 518/1042] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Aug 3 04:55:35 2022] GET /identity/v3/endpoints?service_id=3adf4162784c42dea7f1005f9954b7b5&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:35.263552 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7919699c-1454-4136-9752-09fb6bdb85a0 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:35.308274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:35.314824 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471e393-92ab-4025-9310-be62808a2581 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:35.350912 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b8696-3525-4044-a9d0-e01cd97a7a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:35.351924 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b8696-3525-4044-a9d0-e01cd97a7a59 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:35.352309 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b8696-3525-4044-a9d0-e01cd97a7a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:35.353041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b8696-3525-4044-a9d0-e01cd97a7a59 None admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:35.353897 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b8696-3525-4044-a9d0-e01cd97a7a59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:35.354317 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b8696-3525-4044-a9d0-e01cd97a7a59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:35.356067 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b8696-3525-4044-a9d0-e01cd97a7a59 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:35.370602 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 525/1043] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:55:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:35.371755 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7919699c-1454-4136-9752-09fb6bdb85a0 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:35.376878 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b8696-3525-4044-a9d0-e01cd97a7a59 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:35.418651 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44192048-a035-4b3f-abae-5e88e4166441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:35.420006 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44192048-a035-4b3f-abae-5e88e4166441 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:35.421418 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44192048-a035-4b3f-abae-5e88e4166441 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:35.421958 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44192048-a035-4b3f-abae-5e88e4166441 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:35.423143 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44192048-a035-4b3f-abae-5e88e4166441 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:35.425545 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44192048-a035-4b3f-abae-5e88e4166441 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:35.446974 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 519/1044] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Aug 3 04:55:35 2022] GET /identity/v3/services => generated 1957 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 04:55:35.449006 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:35.521832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b8696-3525-4044-a9d0-e01cd97a7a59 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:35.532489 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afacec1-0476-427d-92b3-7557cab215da None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:35.532790 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afacec1-0476-427d-92b3-7557cab215da None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:35.533075 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afacec1-0476-427d-92b3-7557cab215da None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:35.534719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 520/1045] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:55:35.540325 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab58c13-2911-4b0b-8272-ca866ab030c9 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:35.540617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab58c13-2911-4b0b-8272-ca866ab030c9 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:35.540901 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab58c13-2911-4b0b-8272-ca866ab030c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:35.560780 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-1ab58c13-2911-4b0b-8272-ca866ab030c9 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:35.562013 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1ab58c13-2911-4b0b-8272-ca866ab030c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:35.563181 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fada8ce-8703-42fd-a430-7ed480323b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:35.564009 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fada8ce-8703-42fd-a430-7ed480323b49 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:35.564464 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fada8ce-8703-42fd-a430-7ed480323b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:35.564951 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fada8ce-8703-42fd-a430-7ed480323b49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:35.565920 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fada8ce-8703-42fd-a430-7ed480323b49 None admin] RBAC: Adding query filter params (role.id=729f6ba7aded468788a72fefb5763fc4, scope.project.id=cf3763a37383456184a9ca9351b4ce03, user.id=a3fa4a6c16274c18849869b55b33ac62) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:35.566239 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fada8ce-8703-42fd-a430-7ed480323b49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:35.567718 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fada8ce-8703-42fd-a430-7ed480323b49 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:35.582999 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 526/1046] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:55:35 2022] GET /identity/v3/role_assignments?user.id=a3fa4a6c16274c18849869b55b33ac62&scope.project.id=cf3763a37383456184a9ca9351b4ce03&role.id=729f6ba7aded468788a72fefb5763fc4 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:35.650493 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 521/1047] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:35 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 04:55:35.674123 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fada8ce-8703-42fd-a430-7ed480323b49 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:35.782332 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:35.785323 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fada8ce-8703-42fd-a430-7ed480323b49 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:35.820289 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ad778-b02b-41ee-af61-d6cb4e365ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:35.820942 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ad778-b02b-41ee-af61-d6cb4e365ebb None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:35.821227 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ad778-b02b-41ee-af61-d6cb4e365ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:35.821505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ad778-b02b-41ee-af61-d6cb4e365ebb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:35.832005 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ad778-b02b-41ee-af61-d6cb4e365ebb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:55:35.837353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-4f5ad778-b02b-41ee-af61-d6cb4e365ebb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:55:35.837353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:35.837353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:35.837353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:35.837353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:35.837353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:35.837353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:35.837353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:35.837353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:35.837353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:35.837353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:35.837353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:35.837353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:55:35.837353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:55:35.837353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:55:35.837353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:55:35.838255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:55:35.838255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:55:35.838255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:55:35.838255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:55:35.838255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:35.838255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:35.838255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:35.838255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:35.838255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:35.838255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:35.838255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:35.838255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:35.838255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:35.838255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:35.838255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:35.838255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:35.839478 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:35.839478 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:35.839478 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:35.839478 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:35.839478 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:55:35.839478 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:55:35.839478 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:55:35.839478 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:55:35.839478 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:55:35.839478 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:55:35.839478 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:55:35.839478 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:35.842910 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 527/1048] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:55:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:35.845310 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab58c13-2911-4b0b-8272-ca866ab030c9 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:35.880386 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:35.950111 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab58c13-2911-4b0b-8272-ca866ab030c9 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:35.987174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd94c42-271c-408e-a276-b024ad414f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:35.987811 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd94c42-271c-408e-a276-b024ad414f68 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:35.988096 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd94c42-271c-408e-a276-b024ad414f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:35.988378 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd94c42-271c-408e-a276-b024ad414f68 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:35.989097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd94c42-271c-408e-a276-b024ad414f68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:35.989406 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd94c42-271c-408e-a276-b024ad414f68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:35.990782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd94c42-271c-408e-a276-b024ad414f68 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:36.004021 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 522/1049] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:55:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:36.005611 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ad778-b02b-41ee-af61-d6cb4e365ebb None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:36.112054 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ad778-b02b-41ee-af61-d6cb4e365ebb None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:36.146750 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f860aa-91e5-4b38-9177-1eeee7e96be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:36.147441 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f860aa-91e5-4b38-9177-1eeee7e96be7 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:36.147688 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f860aa-91e5-4b38-9177-1eeee7e96be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:36.147966 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f860aa-91e5-4b38-9177-1eeee7e96be7 None admin] PATH_INFO: `/v3/users/68e8bdea8e0d4b0183d66d2e05056e0b` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:36.162280 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f860aa-91e5-4b38-9177-1eeee7e96be7 None admin] RBAC: Authorizing `identity:get_user(user_id=68e8bdea8e0d4b0183d66d2e05056e0b)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:36.163837 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f860aa-91e5-4b38-9177-1eeee7e96be7 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:36.170719 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 528/1050] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:55:36 2022] GET /identity/v3/users/68e8bdea8e0d4b0183d66d2e05056e0b => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:36.172247 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd94c42-271c-408e-a276-b024ad414f68 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:36.268291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:36.275554 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd94c42-271c-408e-a276-b024ad414f68 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:36.309531 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa175cab-04fc-46fb-97c2-6b1251029ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:36.310160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa175cab-04fc-46fb-97c2-6b1251029ecb None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:36.310499 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa175cab-04fc-46fb-97c2-6b1251029ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:36.310784 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa175cab-04fc-46fb-97c2-6b1251029ecb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:36.380280 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa175cab-04fc-46fb-97c2-6b1251029ecb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:36.381666 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa175cab-04fc-46fb-97c2-6b1251029ecb None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:36.479012 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:36.491571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 523/1051] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:55:36 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 03 04:55:36.497281 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f860aa-91e5-4b38-9177-1eeee7e96be7 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:36.517300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:36.603536 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f860aa-91e5-4b38-9177-1eeee7e96be7 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:36.638959 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65054d9-4a6b-42b1-8cde-b063df39faca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:36.639527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65054d9-4a6b-42b1-8cde-b063df39faca None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:36.639813 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65054d9-4a6b-42b1-8cde-b063df39faca None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:36.640094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65054d9-4a6b-42b1-8cde-b063df39faca None admin] PATH_INFO: `/v3/projects/e1dd3a50e6ca4b0b91e9aadd4fb66d34` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:36.641301 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65054d9-4a6b-42b1-8cde-b063df39faca None admin] RBAC: Authorizing `identity:get_project(project_id=e1dd3a50e6ca4b0b91e9aadd4fb66d34)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:36.642751 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65054d9-4a6b-42b1-8cde-b063df39faca None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:36.644466 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 529/1052] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:55:36 2022] GET /identity/v3/projects/e1dd3a50e6ca4b0b91e9aadd4fb66d34 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:36.651073 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa175cab-04fc-46fb-97c2-6b1251029ecb None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:36.757289 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa175cab-04fc-46fb-97c2-6b1251029ecb None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:36.791838 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95805f9-227a-485d-8070-a0d8865efd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:36.792455 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95805f9-227a-485d-8070-a0d8865efd3e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:36.792762 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95805f9-227a-485d-8070-a0d8865efd3e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:36.793040 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95805f9-227a-485d-8070-a0d8865efd3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:36.793818 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95805f9-227a-485d-8070-a0d8865efd3e None admin] RBAC: Adding query filter params (role.id=e39d4ecea4c64d5794e1079c9be3ad36, scope.project.id=e1dd3a50e6ca4b0b91e9aadd4fb66d34, user.id=68e8bdea8e0d4b0183d66d2e05056e0b) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:36.794128 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95805f9-227a-485d-8070-a0d8865efd3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:36.795606 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95805f9-227a-485d-8070-a0d8865efd3e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:36.807098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 524/1053] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:55:36 2022] GET /identity/v3/role_assignments?user.id=68e8bdea8e0d4b0183d66d2e05056e0b&scope.project.id=e1dd3a50e6ca4b0b91e9aadd4fb66d34&role.id=e39d4ecea4c64d5794e1079c9be3ad36 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:37.485138 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12aadc1-9aae-4239-b3d8-1022d9d131a4 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:37.485701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12aadc1-9aae-4239-b3d8-1022d9d131a4 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:37.485701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12aadc1-9aae-4239-b3d8-1022d9d131a4 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:37.487083 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 530/1054] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:37.492431 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d11fc2-33ec-4f47-9cf5-b67b71bc8f14 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:37.492834 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d11fc2-33ec-4f47-9cf5-b67b71bc8f14 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:37.493016 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d11fc2-33ec-4f47-9cf5-b67b71bc8f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:37.510991 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-12d11fc2-33ec-4f47-9cf5-b67b71bc8f14 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:37.512203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-12d11fc2-33ec-4f47-9cf5-b67b71bc8f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:37.570057 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:37.581535 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 525/1055] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 04:55:37.604290 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12aadc1-9aae-4239-b3d8-1022d9d131a4 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:37.614398 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e59a24-c289-4f3b-8ce6-839963aa5322 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:37.614692 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e59a24-c289-4f3b-8ce6-839963aa5322 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:37.615006 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e59a24-c289-4f3b-8ce6-839963aa5322 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:37.616493 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 526/1056] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 04:55:37.621173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd1061-ea27-45ab-88b2-8a46ae17b223 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:37.621462 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd1061-ea27-45ab-88b2-8a46ae17b223 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:37.621747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd1061-ea27-45ab-88b2-8a46ae17b223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:37.639425 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-84dd1061-ea27-45ab-88b2-8a46ae17b223 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:37.640627 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-84dd1061-ea27-45ab-88b2-8a46ae17b223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:37.700435 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:37.714058 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 527/1057] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 04:55:37.718324 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12aadc1-9aae-4239-b3d8-1022d9d131a4 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:37.737466 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd1061-ea27-45ab-88b2-8a46ae17b223 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:37.758025 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ce20a-6dac-4187-86f2-0f8d4e67bbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:37.758677 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ce20a-6dac-4187-86f2-0f8d4e67bbde None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:37.758996 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ce20a-6dac-4187-86f2-0f8d4e67bbde None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:37.759268 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ce20a-6dac-4187-86f2-0f8d4e67bbde None admin] PATH_INFO: `/v3/services/image` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:37.759894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ce20a-6dac-4187-86f2-0f8d4e67bbde None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:37.761262 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ce20a-6dac-4187-86f2-0f8d4e67bbde None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:37.771542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-8a8ce20a-6dac-4187-86f2-0f8d4e67bbde None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 04:55:37.771542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:37.771542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 04:55:37.771542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 04:55:37.771542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 04:55:37.771542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 04:55:37.771542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 04:55:37.771542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:37.771542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 04:55:37.771542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:37.771542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:55:37.771542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:37.771542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:37.771542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:37.771542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:37.771542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:37.771542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:37.772545 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:37.772545 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:37.772545 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:37.772545 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:37.772545 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:37.772545 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:37.772545 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 04:55:37.772545 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 04:55:37.772545 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 04:55:37.772545 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 04:55:37.772545 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:37.772545 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:37.772545 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:37.772545 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:37.772545 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:37.772545 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:37.773482 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:37.773482 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:37.773482 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:37.773482 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:37.773482 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:37.773482 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:37.773482 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:37.773482 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:37.773482 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:37.773482 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:37.773482 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 04:55:37.773482 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:55:37.773482 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 04:55:37.773482 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:37.773482 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 531/1058] 10.222.0.40 () {56 vars in 1105 bytes} [Wed Aug 3 04:55:37 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:37.779176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ce20a-6dac-4187-86f2-0f8d4e67bbde None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:37.844938 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd1061-ea27-45ab-88b2-8a46ae17b223 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:37.866256 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:37.883981 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ce20a-6dac-4187-86f2-0f8d4e67bbde None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:37.884221 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89608a7e-5143-40a5-bbe8-6267c9e12496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:37.884778 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89608a7e-5143-40a5-bbe8-6267c9e12496 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:37.885063 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89608a7e-5143-40a5-bbe8-6267c9e12496 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:37.885341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89608a7e-5143-40a5-bbe8-6267c9e12496 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:37.898103 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89608a7e-5143-40a5-bbe8-6267c9e12496 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:55:37.903529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-89608a7e-5143-40a5-bbe8-6267c9e12496 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 04:55:37.903529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:37.903529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:37.903529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:37.903529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:37.903529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:37.903529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:37.903529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:37.903529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:37.903529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:37.903529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:37.903529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:37.903529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:55:37.903529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:55:37.903529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:55:37.903529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:55:37.904471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:55:37.904471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:55:37.904471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:55:37.904471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:55:37.904471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:37.904471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:37.904471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:37.904471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:37.904471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:37.904471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:37.904471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:37.904471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:37.904471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:37.904471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:37.904471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:37.904471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:37.905319 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:37.905319 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:37.905319 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:37.905319 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:37.905319 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:55:37.905319 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:55:37.905319 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:55:37.905319 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:55:37.905319 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:55:37.905319 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:55:37.905319 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 04:55:37.905319 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:37.905319 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 528/1059] 10.222.0.40 () {56 vars in 1120 bytes} [Wed Aug 3 04:55:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:55:37.910813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89608a7e-5143-40a5-bbe8-6267c9e12496 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:37.928000 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9651186-3542-47c8-a41c-ca29ec969bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:37.928638 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9651186-3542-47c8-a41c-ca29ec969bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:37.928916 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9651186-3542-47c8-a41c-ca29ec969bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:37.929195 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9651186-3542-47c8-a41c-ca29ec969bb7 None admin] PATH_INFO: `/v3/services` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:37.929893 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9651186-3542-47c8-a41c-ca29ec969bb7 None admin] RBAC: Adding query filter params (name=image) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:37.930173 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9651186-3542-47c8-a41c-ca29ec969bb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:37.931587 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9651186-3542-47c8-a41c-ca29ec969bb7 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:37.941912 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 532/1060] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:37.942588 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9651186-3542-47c8-a41c-ca29ec969bb7 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:37.959173 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:38.023422 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89608a7e-5143-40a5-bbe8-6267c9e12496 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:38.051550 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9651186-3542-47c8-a41c-ca29ec969bb7 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:38.061531 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6423a-dfad-4b8a-b656-6f90dfb3e752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:38.062470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6423a-dfad-4b8a-b656-6f90dfb3e752 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:38.062938 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6423a-dfad-4b8a-b656-6f90dfb3e752 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:38.063222 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6423a-dfad-4b8a-b656-6f90dfb3e752 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:38.063950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf6423a-dfad-4b8a-b656-6f90dfb3e752 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:38.064264 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf6423a-dfad-4b8a-b656-6f90dfb3e752 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:38.065640 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf6423a-dfad-4b8a-b656-6f90dfb3e752 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:38.074495 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 529/1061] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Aug 3 04:55:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:38.079457 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6423a-dfad-4b8a-b656-6f90dfb3e752 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:38.090739 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb695a-ecb1-4683-a5cc-d07eeb6f2cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:38.091596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb695a-ecb1-4683-a5cc-d07eeb6f2cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:38.091880 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb695a-ecb1-4683-a5cc-d07eeb6f2cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:38.092160 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb695a-ecb1-4683-a5cc-d07eeb6f2cc3 None admin] PATH_INFO: `/v3/services` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:38.092835 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bb695a-ecb1-4683-a5cc-d07eeb6f2cc3 None admin] RBAC: Adding query filter params (type=image) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:38.093144 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bb695a-ecb1-4683-a5cc-d07eeb6f2cc3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:38.094551 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bb695a-ecb1-4683-a5cc-d07eeb6f2cc3 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:38.099373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 533/1062] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:37 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:38.104892 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb695a-ecb1-4683-a5cc-d07eeb6f2cc3 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:38.125529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:38.184912 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6423a-dfad-4b8a-b656-6f90dfb3e752 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:38.194353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:38.215757 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb695a-ecb1-4683-a5cc-d07eeb6f2cc3 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:38.220562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457711a-6b3c-4bba-89ac-6316426f98db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:38.221348 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457711a-6b3c-4bba-89ac-6316426f98db None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:38.221535 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457711a-6b3c-4bba-89ac-6316426f98db None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:38.221762 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457711a-6b3c-4bba-89ac-6316426f98db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:38.233817 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c457711a-6b3c-4bba-89ac-6316426f98db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:38.235312 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c457711a-6b3c-4bba-89ac-6316426f98db None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:38.250333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-c457711a-6b3c-4bba-89ac-6316426f98db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:38.250333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:38.250333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:38.250333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:38.250333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:38.250333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:38.250333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:38.250333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:38.250333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:38.250333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:38.250333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:38.250333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:38.250333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:55:38.250333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:55:38.250333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:55:38.251288 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:55:38.251288 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:38.251288 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:38.251288 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:38.251288 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:38.251288 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:38.251288 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:38.251288 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:38.251288 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:38.251288 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:38.251288 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:38.251288 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:38.251288 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:38.251288 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:38.251288 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:38.251288 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:38.251288 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:38.252563 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:55:38.252563 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:55:38.252563 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:38.252563 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:38.252563 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 530/1063] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:55:38.254485 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b5ba5-09a2-40b3-b320-2e7dd2393dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:38.255134 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b5ba5-09a2-40b3-b320-2e7dd2393dc0 None admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:38.255415 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b5ba5-09a2-40b3-b320-2e7dd2393dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:38.255687 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b5ba5-09a2-40b3-b320-2e7dd2393dc0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:38.256389 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4b5ba5-09a2-40b3-b320-2e7dd2393dc0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:38.257546 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457711a-6b3c-4bba-89ac-6316426f98db None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:38.257852 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4b5ba5-09a2-40b3-b320-2e7dd2393dc0 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:38.340014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 534/1064] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 04:55:38 2022] POST /identity/v3/endpoints => generated 327 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 04:55:38.356429 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:38.363931 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457711a-6b3c-4bba-89ac-6316426f98db None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:38.438196 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e4af4-d50b-43cb-9189-d6da2e2d9860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:38.438931 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e4af4-d50b-43cb-9189-d6da2e2d9860 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:38.439222 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e4af4-d50b-43cb-9189-d6da2e2d9860 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:38.439501 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e4af4-d50b-43cb-9189-d6da2e2d9860 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:38.440215 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e4af4-d50b-43cb-9189-d6da2e2d9860 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:38.440526 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e4af4-d50b-43cb-9189-d6da2e2d9860 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:38.441911 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e4af4-d50b-43cb-9189-d6da2e2d9860 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:38.455730 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 531/1065] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:55:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:38.461296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b5ba5-09a2-40b3-b320-2e7dd2393dc0 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:38.575820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b5ba5-09a2-40b3-b320-2e7dd2393dc0 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:38.612759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8b8c0-f3db-4dc8-ad6d-2623e3498f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:38.613831 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8b8c0-f3db-4dc8-ad6d-2623e3498f9e None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:38.614326 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8b8c0-f3db-4dc8-ad6d-2623e3498f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:38.614806 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8b8c0-f3db-4dc8-ad6d-2623e3498f9e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:38.617134 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:38.624198 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef8b8c0-f3db-4dc8-ad6d-2623e3498f9e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:38.625676 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef8b8c0-f3db-4dc8-ad6d-2623e3498f9e None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:38.637210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-fef8b8c0-f3db-4dc8-ad6d-2623e3498f9e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 04:55:38.637210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:38.637210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:38.637210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:38.637210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:38.637210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:38.637210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:38.637210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:38.637210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:38.637210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:38.637210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:38.637210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:38.637210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:55:38.637210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:55:38.637210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:55:38.637210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:55:38.638177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:38.638177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:38.638177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:55:38.638177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:55:38.638177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:55:38.638177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:55:38.638177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:38.638177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:38.638177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:38.638177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:38.638177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:38.638177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:38.638177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:38.638177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:38.638177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:38.638177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:38.638177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:38.638177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:38.639577 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:38.639577 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:38.639577 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:55:38.639577 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:55:38.639577 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:55:38.639577 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:55:38.639577 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:55:38.639577 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:55:38.639577 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 04:55:38.639577 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:38.639577 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 535/1066] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Aug 3 04:55:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:38.644520 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e4af4-d50b-43cb-9189-d6da2e2d9860 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:38.754536 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e4af4-d50b-43cb-9189-d6da2e2d9860 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:38.804696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424dfaa8-f8c3-41cd-aad9-77d7f804175b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:38.805328 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424dfaa8-f8c3-41cd-aad9-77d7f804175b None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:38.805608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424dfaa8-f8c3-41cd-aad9-77d7f804175b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:38.805888 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424dfaa8-f8c3-41cd-aad9-77d7f804175b None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:38.806690 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424dfaa8-f8c3-41cd-aad9-77d7f804175b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:38.807029 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424dfaa8-f8c3-41cd-aad9-77d7f804175b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:38.808498 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424dfaa8-f8c3-41cd-aad9-77d7f804175b None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:38.824607 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf06cbe-b708-4989-a236-664f221a69e6 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:38.824922 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf06cbe-b708-4989-a236-664f221a69e6 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:38.825175 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf06cbe-b708-4989-a236-664f221a69e6 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:38.826811 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 536/1067] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:38.827655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 532/1068] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Aug 3 04:55:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:38.832055 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416de716-617a-4c85-84f4-88268a49c087 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:38.832513 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416de716-617a-4c85-84f4-88268a49c087 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:38.832802 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416de716-617a-4c85-84f4-88268a49c087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:38.833185 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424dfaa8-f8c3-41cd-aad9-77d7f804175b None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:38.844394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:38.853066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-416de716-617a-4c85-84f4-88268a49c087 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:38.854486 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-416de716-617a-4c85-84f4-88268a49c087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:38.932781 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 537/1069] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:55:38.938825 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424dfaa8-f8c3-41cd-aad9-77d7f804175b None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:38.954490 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416de716-617a-4c85-84f4-88268a49c087 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:38.976570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05c98c8-852a-479f-88c3-71f0518d9ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:38.977216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c98c8-852a-479f-88c3-71f0518d9ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:38.977500 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c98c8-852a-479f-88c3-71f0518d9ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:38.977780 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c98c8-852a-479f-88c3-71f0518d9ad5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:38.988404 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05c98c8-852a-479f-88c3-71f0518d9ad5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:38.989857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05c98c8-852a-479f-88c3-71f0518d9ad5 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:39.004647 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-a05c98c8-852a-479f-88c3-71f0518d9ad5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:39.004647 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:39.004647 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:39.004647 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:39.004647 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:39.004647 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:39.004647 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:39.004647 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:39.004647 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:39.004647 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:39.004647 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:39.004647 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:39.004647 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:55:39.004647 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:55:39.004647 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:55:39.005777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:55:39.005777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:39.005777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:39.005777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:39.005777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:39.005777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:39.005777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:39.005777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:39.005777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:39.005777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:39.005777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:39.005777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:39.005777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:39.005777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:39.005777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:39.005777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:39.005777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:39.006834 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:55:39.006834 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:55:39.006834 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:39.006834 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:39.006834 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 533/1070] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:39.011415 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05c98c8-852a-479f-88c3-71f0518d9ad5 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:39.060371 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416de716-617a-4c85-84f4-88268a49c087 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:39.073083 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:39.085473 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:39.104389 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3985252-4a8f-4d9b-820c-4226ee77519b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:39.105060 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3985252-4a8f-4d9b-820c-4226ee77519b None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:39.105375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3985252-4a8f-4d9b-820c-4226ee77519b None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:39.105651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3985252-4a8f-4d9b-820c-4226ee77519b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:39.118589 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3985252-4a8f-4d9b-820c-4226ee77519b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:55:39.123003 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05c98c8-852a-479f-88c3-71f0518d9ad5 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:39.123993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-f3985252-4a8f-4d9b-820c-4226ee77519b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:55:39.123993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:39.123993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:39.123993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:39.123993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:39.123993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:39.123993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:39.123993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:39.123993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:39.123993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:39.123993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:39.123993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:39.123993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:55:39.123993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:55:39.123993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:55:39.123993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:55:39.124932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:55:39.124932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:55:39.124932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:55:39.124932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:55:39.124932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:39.124932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:39.124932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:39.124932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:39.124932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:39.124932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:39.124932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:39.124932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:39.124932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:39.124932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:39.124932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:39.124932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:39.125792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:39.125792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:39.125792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:39.125792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:39.125792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:55:39.125792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:55:39.125792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:55:39.125792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:55:39.125792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:55:39.125792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:55:39.125792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:55:39.125792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:39.125792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 538/1071] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:55:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:55:39.131225 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3985252-4a8f-4d9b-820c-4226ee77519b None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:39.164324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243bde1b-2c8d-49d8-8b0d-d9b5fd2b1cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:39.164947 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bde1b-2c8d-49d8-8b0d-d9b5fd2b1cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:39.165227 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bde1b-2c8d-49d8-8b0d-d9b5fd2b1cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:39.165498 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bde1b-2c8d-49d8-8b0d-d9b5fd2b1cd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:39.166190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243bde1b-2c8d-49d8-8b0d-d9b5fd2b1cd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:39.166497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243bde1b-2c8d-49d8-8b0d-d9b5fd2b1cd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:39.168487 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243bde1b-2c8d-49d8-8b0d-d9b5fd2b1cd1 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:39.181733 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 534/1072] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:55:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:39.187347 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243bde1b-2c8d-49d8-8b0d-d9b5fd2b1cd1 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:39.242939 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3985252-4a8f-4d9b-820c-4226ee77519b None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:39.280752 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c576b-3530-4bc7-90d1-2873e814f535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:39.281538 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c576b-3530-4bc7-90d1-2873e814f535 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:39.281822 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c576b-3530-4bc7-90d1-2873e814f535 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:39.282111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c576b-3530-4bc7-90d1-2873e814f535 None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:39.282809 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c576b-3530-4bc7-90d1-2873e814f535 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:39.283146 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c576b-3530-4bc7-90d1-2873e814f535 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:39.284797 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c576b-3530-4bc7-90d1-2873e814f535 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:39.293688 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 539/1073] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:55:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:39.296122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243bde1b-2c8d-49d8-8b0d-d9b5fd2b1cd1 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:39.299252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c576b-3530-4bc7-90d1-2873e814f535 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:39.320816 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:39.338902 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc0328-0e99-4ebc-8205-d14cb98e3919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:39.340158 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc0328-0e99-4ebc-8205-d14cb98e3919 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:39.340401 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc0328-0e99-4ebc-8205-d14cb98e3919 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:39.340679 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc0328-0e99-4ebc-8205-d14cb98e3919 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:39.347650 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc0328-0e99-4ebc-8205-d14cb98e3919 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:39.348393 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:39.349499 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc0328-0e99-4ebc-8205-d14cb98e3919 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:39.362138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-01dc0328-0e99-4ebc-8205-d14cb98e3919 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:55:39.362138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:39.362138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:39.362138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:39.362138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:39.362138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:39.362138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:39.362138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:39.362138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:39.362138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:39.362138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:39.362138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:39.362138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:55:39.362138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:55:39.362138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:55:39.363127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:55:39.363127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:39.363127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:39.363127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:39.363127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:39.363127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:39.363127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:39.363127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:39.363127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:39.363127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:39.363127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:39.363127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:39.363127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:39.363127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:39.363127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:39.363127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:39.363127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:39.364096 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:55:39.364096 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:55:39.364096 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:55:39.364096 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:55:39.364096 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:55:39.364096 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:55:39.364096 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:55:39.364096 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:39.364096 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 535/1074] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:55:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:55:39.369142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc0328-0e99-4ebc-8205-d14cb98e3919 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:39.409642 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c576b-3530-4bc7-90d1-2873e814f535 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:39.444732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0936989b-805d-4a38-a2a5-5bb061344d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:39.445292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936989b-805d-4a38-a2a5-5bb061344d91 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:39.445572 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936989b-805d-4a38-a2a5-5bb061344d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:39.445847 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936989b-805d-4a38-a2a5-5bb061344d91 None admin] PATH_INFO: `/v3/users/68e8bdea8e0d4b0183d66d2e05056e0b` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:39.447114 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0936989b-805d-4a38-a2a5-5bb061344d91 None admin] RBAC: Authorizing `identity:get_user(user_id=68e8bdea8e0d4b0183d66d2e05056e0b)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:39.448551 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0936989b-805d-4a38-a2a5-5bb061344d91 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:39.457649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 540/1075] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:55:39 2022] GET /identity/v3/users/68e8bdea8e0d4b0183d66d2e05056e0b => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:39.458493 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0936989b-805d-4a38-a2a5-5bb061344d91 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:39.487402 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc0328-0e99-4ebc-8205-d14cb98e3919 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:39.523487 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e5fac-8027-4b22-a4f6-5dda2989c381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:39.524133 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e5fac-8027-4b22-a4f6-5dda2989c381 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:39.524417 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e5fac-8027-4b22-a4f6-5dda2989c381 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:39.524700 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e5fac-8027-4b22-a4f6-5dda2989c381 None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:39.525472 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e5fac-8027-4b22-a4f6-5dda2989c381 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:39.525782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e5fac-8027-4b22-a4f6-5dda2989c381 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:39.527859 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e5fac-8027-4b22-a4f6-5dda2989c381 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:39.541457 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 536/1076] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:55:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:39.547221 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e5fac-8027-4b22-a4f6-5dda2989c381 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:39.581781 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0936989b-805d-4a38-a2a5-5bb061344d91 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:39.592914 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:39.597198 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:39.618659 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d1de5-93f6-4b1b-a24f-9b0dad9c86d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:39.619327 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d1de5-93f6-4b1b-a24f-9b0dad9c86d5 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:39.619652 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d1de5-93f6-4b1b-a24f-9b0dad9c86d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:39.619873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d1de5-93f6-4b1b-a24f-9b0dad9c86d5 None admin] PATH_INFO: `/v3/projects/e1dd3a50e6ca4b0b91e9aadd4fb66d34` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:39.621543 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106d1de5-93f6-4b1b-a24f-9b0dad9c86d5 None admin] RBAC: Authorizing `identity:get_project(project_id=e1dd3a50e6ca4b0b91e9aadd4fb66d34)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:39.623011 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106d1de5-93f6-4b1b-a24f-9b0dad9c86d5 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:39.625207 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 541/1077] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:55:39 2022] GET /identity/v3/projects/e1dd3a50e6ca4b0b91e9aadd4fb66d34 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:39.630349 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d1de5-93f6-4b1b-a24f-9b0dad9c86d5 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:39.667873 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e5fac-8027-4b22-a4f6-5dda2989c381 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:39.704077 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6099a45-8f97-43c8-9dcd-82c8305dc3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:39.704670 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6099a45-8f97-43c8-9dcd-82c8305dc3d5 None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:39.704955 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6099a45-8f97-43c8-9dcd-82c8305dc3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:39.705258 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6099a45-8f97-43c8-9dcd-82c8305dc3d5 None admin] PATH_INFO: `/v3/projects/cf3763a37383456184a9ca9351b4ce03/users/a3fa4a6c16274c18849869b55b33ac62/roles/729f6ba7aded468788a72fefb5763fc4` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:39.707505 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6099a45-8f97-43c8-9dcd-82c8305dc3d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf3763a37383456184a9ca9351b4ce03, user_id=a3fa4a6c16274c18849869b55b33ac62, role_id=729f6ba7aded468788a72fefb5763fc4)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:39.708942 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6099a45-8f97-43c8-9dcd-82c8305dc3d5 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:39.743307 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d1de5-93f6-4b1b-a24f-9b0dad9c86d5 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:39.777594 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297ec00-1715-4c7f-ac03-5fa6d0122862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:39.778269 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297ec00-1715-4c7f-ac03-5fa6d0122862 None admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:39.778556 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297ec00-1715-4c7f-ac03-5fa6d0122862 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:39.779020 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297ec00-1715-4c7f-ac03-5fa6d0122862 None admin] PATH_INFO: `/v3/projects/e1dd3a50e6ca4b0b91e9aadd4fb66d34/users/68e8bdea8e0d4b0183d66d2e05056e0b/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:39.780804 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3297ec00-1715-4c7f-ac03-5fa6d0122862 None admin] RBAC: Authorizing `identity:create_grant(project_id=e1dd3a50e6ca4b0b91e9aadd4fb66d34, user_id=68e8bdea8e0d4b0183d66d2e05056e0b, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:39.782405 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3297ec00-1715-4c7f-ac03-5fa6d0122862 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:39.783535 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 537/1078] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 3 04:55:39 2022] PUT /identity/v3/projects/cf3763a37383456184a9ca9351b4ce03/users/a3fa4a6c16274c18849869b55b33ac62/roles/729f6ba7aded468788a72fefb5763fc4 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 04:55:39.881162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 542/1079] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 3 04:55:39 2022] PUT /identity/v3/projects/e1dd3a50e6ca4b0b91e9aadd4fb66d34/users/68e8bdea8e0d4b0183d66d2e05056e0b/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 04:55:40.432058 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df52b5d-af9d-44a4-bf5c-604a3ee6b437 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:40.432344 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df52b5d-af9d-44a4-bf5c-604a3ee6b437 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:40.432626 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df52b5d-af9d-44a4-bf5c-604a3ee6b437 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:40.434014 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 538/1080] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:40.439159 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db1c96-1d6a-4849-894f-69272c84a953 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:40.439451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db1c96-1d6a-4849-894f-69272c84a953 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:40.439806 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db1c96-1d6a-4849-894f-69272c84a953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:40.457530 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-f8db1c96-1d6a-4849-894f-69272c84a953 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:40.458715 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-f8db1c96-1d6a-4849-894f-69272c84a953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:40.495279 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:40.538339 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 543/1081] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:55:40.561405 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df52b5d-af9d-44a4-bf5c-604a3ee6b437 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:40.670194 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:40.674225 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df52b5d-af9d-44a4-bf5c-604a3ee6b437 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:40.708522 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f118a18-e17f-4ebd-8ec4-a86db3444e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:40.709080 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f118a18-e17f-4ebd-8ec4-a86db3444e53 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:40.709346 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f118a18-e17f-4ebd-8ec4-a86db3444e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:40.709637 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f118a18-e17f-4ebd-8ec4-a86db3444e53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:40.780125 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f118a18-e17f-4ebd-8ec4-a86db3444e53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:40.781638 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f118a18-e17f-4ebd-8ec4-a86db3444e53 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:40.878761 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:40.888720 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 539/1082] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:55:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 04:55:40.890560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db1c96-1d6a-4849-894f-69272c84a953 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:40.994240 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db1c96-1d6a-4849-894f-69272c84a953 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:41.038527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd24237-0fde-419c-a5e0-495c4c43998a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:41.039201 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd24237-0fde-419c-a5e0-495c4c43998a None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:41.039481 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd24237-0fde-419c-a5e0-495c4c43998a None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:41.039752 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd24237-0fde-419c-a5e0-495c4c43998a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:41.049835 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd24237-0fde-419c-a5e0-495c4c43998a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:41.051357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd24237-0fde-419c-a5e0-495c4c43998a None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:41.066285 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-fdd24237-0fde-419c-a5e0-495c4c43998a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:41.066285 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:41.066285 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:41.066285 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:41.066285 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:41.066285 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:41.066285 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:41.066285 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:41.066285 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:41.066285 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:41.066285 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:41.066285 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:41.066285 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:55:41.066285 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:55:41.066285 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:55:41.067224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:55:41.067224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:41.067224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:41.067224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:41.067224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:41.067224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:41.067224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:41.067224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:41.067224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:41.067224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:41.067224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:41.067224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:41.067224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:41.067224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:41.067224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:41.067224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:41.067224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:41.073138 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:55:41.073138 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:55:41.073138 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:41.073138 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:41.073138 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 544/1083] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:55:41.075518 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f118a18-e17f-4ebd-8ec4-a86db3444e53 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:41.177491 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f118a18-e17f-4ebd-8ec4-a86db3444e53 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:41.211821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfaa4df-e7b0-449d-a341-d673d89bf40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:41.212434 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfaa4df-e7b0-449d-a341-d673d89bf40c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:41.212719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfaa4df-e7b0-449d-a341-d673d89bf40c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:41.212994 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfaa4df-e7b0-449d-a341-d673d89bf40c None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:41.213679 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfaa4df-e7b0-449d-a341-d673d89bf40c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:41.213990 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfaa4df-e7b0-449d-a341-d673d89bf40c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:41.215441 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfaa4df-e7b0-449d-a341-d673d89bf40c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:41.226948 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 540/1084] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:55:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:41.798809 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d5d6c-8ecb-4ce2-a6ff-3583168fd06b None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:41.799123 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d5d6c-8ecb-4ce2-a6ff-3583168fd06b None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:41.799404 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d5d6c-8ecb-4ce2-a6ff-3583168fd06b None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:41.800779 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 545/1085] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:41.806037 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe840db-15ae-4f96-829b-8b54af6abf35 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:41.806508 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe840db-15ae-4f96-829b-8b54af6abf35 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:41.806601 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe840db-15ae-4f96-829b-8b54af6abf35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:41.824451 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-4fe840db-15ae-4f96-829b-8b54af6abf35 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:41.825636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-4fe840db-15ae-4f96-829b-8b54af6abf35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:41.880827 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:41.892020 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 541/1086] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:55:41.912633 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c4383-6393-4ce1-bc42-2d8440271e87 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:41.913128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c4383-6393-4ce1-bc42-2d8440271e87 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:41.913241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c4383-6393-4ce1-bc42-2d8440271e87 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:41.914696 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 546/1087] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:41.915650 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe840db-15ae-4f96-829b-8b54af6abf35 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:41.919948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae9a9f-9933-4d4a-9fed-f886dac1031c None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:41.920243 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae9a9f-9933-4d4a-9fed-f886dac1031c None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:41.920524 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae9a9f-9933-4d4a-9fed-f886dac1031c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:41.937984 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-a3ae9a9f-9933-4d4a-9fed-f886dac1031c None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:41.939229 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-a3ae9a9f-9933-4d4a-9fed-f886dac1031c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:41.956734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:42.005797 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 547/1088] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 04:55:42.015996 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe840db-15ae-4f96-829b-8b54af6abf35 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:42.029230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae9a9f-9933-4d4a-9fed-f886dac1031c None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:42.050960 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3be368-57de-4694-b95b-8ec765e62094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:42.051611 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3be368-57de-4694-b95b-8ec765e62094 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:42.051892 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3be368-57de-4694-b95b-8ec765e62094 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:42.052170 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3be368-57de-4694-b95b-8ec765e62094 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:42.062185 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3be368-57de-4694-b95b-8ec765e62094 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:55:42.067103 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-9a3be368-57de-4694-b95b-8ec765e62094 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 04:55:42.067103 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:42.067103 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:42.067103 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:42.067103 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:42.067103 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:42.067103 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:42.067103 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:42.067103 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:42.067103 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:42.067103 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:42.067103 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:42.067103 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:55:42.067103 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:55:42.067103 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:55:42.067103 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:55:42.068138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:55:42.068138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:55:42.068138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:55:42.068138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:55:42.068138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:42.068138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:42.068138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:42.068138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:42.068138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:42.068138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:42.068138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:42.068138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:42.068138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:42.068138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:42.068138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:42.068138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:42.069004 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:42.069004 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:42.069004 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:42.069004 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:42.069004 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:55:42.069004 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:55:42.069004 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:55:42.069004 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:55:42.069004 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:55:42.069004 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:55:42.069004 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 04:55:42.069004 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:42.069004 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 542/1089] 10.222.0.40 () {56 vars in 1120 bytes} [Wed Aug 3 04:55:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:55:42.074363 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3be368-57de-4694-b95b-8ec765e62094 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:42.123959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:42.130561 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae9a9f-9933-4d4a-9fed-f886dac1031c None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:42.165136 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6335f26a-6bbf-4dac-b852-5b42cd8e8345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:42.165704 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6335f26a-6bbf-4dac-b852-5b42cd8e8345 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:42.165989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6335f26a-6bbf-4dac-b852-5b42cd8e8345 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:42.166261 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6335f26a-6bbf-4dac-b852-5b42cd8e8345 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:42.175413 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3be368-57de-4694-b95b-8ec765e62094 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:42.177193 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6335f26a-6bbf-4dac-b852-5b42cd8e8345 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:55:42.182093 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-6335f26a-6bbf-4dac-b852-5b42cd8e8345 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:55:42.182093 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:42.182093 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:42.182093 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:42.182093 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:42.182093 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:42.182093 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:42.182093 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:42.182093 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:42.182093 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:42.182093 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:42.182093 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:42.182093 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:55:42.182093 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:55:42.182093 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:55:42.182093 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:55:42.183279 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:55:42.183279 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:55:42.183279 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:55:42.183279 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:55:42.183279 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:42.183279 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:42.183279 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:42.183279 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:42.183279 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:42.183279 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:42.183279 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:42.183279 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:42.183279 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:42.183279 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:42.183279 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:42.183279 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:42.184338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:42.184338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:42.184338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:42.184338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:42.184338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:55:42.184338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:55:42.184338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:55:42.184338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:55:42.184338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:55:42.184338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:55:42.184338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:55:42.184338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:42.184338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 548/1090] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:55:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:55:42.189203 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6335f26a-6bbf-4dac-b852-5b42cd8e8345 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:42.205851 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:42.213563 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad3fd4-a754-4a7a-a0ad-5740830ac001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:42.214232 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad3fd4-a754-4a7a-a0ad-5740830ac001 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:42.215136 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad3fd4-a754-4a7a-a0ad-5740830ac001 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:42.215413 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad3fd4-a754-4a7a-a0ad-5740830ac001 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:42.216119 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad3fd4-a754-4a7a-a0ad-5740830ac001 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:42.216420 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad3fd4-a754-4a7a-a0ad-5740830ac001 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:42.217774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad3fd4-a754-4a7a-a0ad-5740830ac001 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:42.230644 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 543/1091] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Aug 3 04:55:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:42.231657 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad3fd4-a754-4a7a-a0ad-5740830ac001 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:42.292751 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6335f26a-6bbf-4dac-b852-5b42cd8e8345 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:42.326941 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a01b7b-5e50-4223-97a0-692a944cb86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:42.327561 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a01b7b-5e50-4223-97a0-692a944cb86b None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:42.327848 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a01b7b-5e50-4223-97a0-692a944cb86b None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:42.328127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a01b7b-5e50-4223-97a0-692a944cb86b None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:42.329447 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a01b7b-5e50-4223-97a0-692a944cb86b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:42.329447 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a01b7b-5e50-4223-97a0-692a944cb86b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:42.330567 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a01b7b-5e50-4223-97a0-692a944cb86b None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:42.333209 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad3fd4-a754-4a7a-a0ad-5740830ac001 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:42.338796 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 549/1092] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:55:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:42.345198 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a01b7b-5e50-4223-97a0-692a944cb86b None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:42.347314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:42.367707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6eba8b-7db1-4be5-8443-5f1b02d62e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:42.368349 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6eba8b-7db1-4be5-8443-5f1b02d62e8e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:42.368635 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6eba8b-7db1-4be5-8443-5f1b02d62e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:42.368914 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6eba8b-7db1-4be5-8443-5f1b02d62e8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:42.379735 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6eba8b-7db1-4be5-8443-5f1b02d62e8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:42.381230 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6eba8b-7db1-4be5-8443-5f1b02d62e8e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:42.396008 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-4c6eba8b-7db1-4be5-8443-5f1b02d62e8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:42.396008 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:42.396008 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:42.396008 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:42.396008 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:42.396008 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:42.396008 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:42.396008 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:42.396008 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:42.396008 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:42.396008 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:42.396008 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:42.396008 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:55:42.396008 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:55:42.396008 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:55:42.397353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:55:42.397353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:42.397353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:42.397353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:42.397353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:42.397353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:42.397353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:42.397353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:42.397353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:42.397353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:42.397353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:42.397353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:42.397353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:42.397353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:42.397353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:42.397353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:42.397353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:42.398312 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:55:42.398312 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:55:42.398312 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:42.398312 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:42.398312 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 544/1093] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:42.403693 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6eba8b-7db1-4be5-8443-5f1b02d62e8e None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:42.424411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:42.446810 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a01b7b-5e50-4223-97a0-692a944cb86b None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:42.481569 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ae2e2-9eec-40a0-98ed-71ca773d92ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:42.482209 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ae2e2-9eec-40a0-98ed-71ca773d92ef None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:42.482492 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ae2e2-9eec-40a0-98ed-71ca773d92ef None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:42.482768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ae2e2-9eec-40a0-98ed-71ca773d92ef None admin] PATH_INFO: `/v3/users/68e8bdea8e0d4b0183d66d2e05056e0b` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:42.483975 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ae2e2-9eec-40a0-98ed-71ca773d92ef None admin] RBAC: Authorizing `identity:get_user(user_id=68e8bdea8e0d4b0183d66d2e05056e0b)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:42.485435 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ae2e2-9eec-40a0-98ed-71ca773d92ef None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:42.487064 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 550/1094] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:55:42 2022] GET /identity/v3/users/68e8bdea8e0d4b0183d66d2e05056e0b => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:42.492712 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ae2e2-9eec-40a0-98ed-71ca773d92ef None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:42.503392 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6eba8b-7db1-4be5-8443-5f1b02d62e8e None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:42.547289 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b8490-5302-417e-8c48-72775fecc5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:42.547939 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b8490-5302-417e-8c48-72775fecc5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:42.548224 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b8490-5302-417e-8c48-72775fecc5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:42.548502 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b8490-5302-417e-8c48-72775fecc5f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:42.549224 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b8490-5302-417e-8c48-72775fecc5f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:42.549532 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b8490-5302-417e-8c48-72775fecc5f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:42.550969 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b8490-5302-417e-8c48-72775fecc5f2 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:42.563394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 545/1095] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:42.569063 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b8490-5302-417e-8c48-72775fecc5f2 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:42.593649 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:42.595544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ae2e2-9eec-40a0-98ed-71ca773d92ef None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:42.629311 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315cbc63-7e85-41a8-a85c-f54674c7a8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:42.629918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315cbc63-7e85-41a8-a85c-f54674c7a8b6 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:42.630202 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315cbc63-7e85-41a8-a85c-f54674c7a8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:42.630479 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315cbc63-7e85-41a8-a85c-f54674c7a8b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:42.647992 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:42.672461 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b8490-5302-417e-8c48-72775fecc5f2 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:42.704572 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315cbc63-7e85-41a8-a85c-f54674c7a8b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:42.706047 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315cbc63-7e85-41a8-a85c-f54674c7a8b6 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:42.707455 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05e9cc-2338-48af-9478-267403f14dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:42.708098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05e9cc-2338-48af-9478-267403f14dfe None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:42.708405 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05e9cc-2338-48af-9478-267403f14dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:42.708688 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05e9cc-2338-48af-9478-267403f14dfe None admin] PATH_INFO: `/v3/users/nova` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:42.715810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb05e9cc-2338-48af-9478-267403f14dfe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:42.717245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb05e9cc-2338-48af-9478-267403f14dfe None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:42.728700 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-fb05e9cc-2338-48af-9478-267403f14dfe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 04:55:42.728700 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:42.728700 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:42.728700 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:42.728700 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:42.728700 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:42.728700 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:42.728700 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:42.728700 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:42.728700 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:42.728700 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:42.728700 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:42.728700 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:55:42.728700 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:55:42.728700 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:55:42.728700 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:55:42.729675 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:42.729675 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:42.729675 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:55:42.729675 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:55:42.729675 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:55:42.729675 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:55:42.729675 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:42.729675 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:42.729675 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:42.729675 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:42.729675 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:42.729675 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:42.729675 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:42.729675 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:42.729675 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:42.729675 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:42.729675 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:42.729675 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:42.730655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:42.730655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:42.730655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:55:42.730655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:55:42.730655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:55:42.730655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:55:42.730655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:55:42.730655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:55:42.730655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 04:55:42.730655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:55:42.730655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 546/1096] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Aug 3 04:55:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:55:42.735723 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05e9cc-2338-48af-9478-267403f14dfe None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:42.807834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 551/1097] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:55:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 03 04:55:42.819818 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315cbc63-7e85-41a8-a85c-f54674c7a8b6 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:42.823177 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:42.846222 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05e9cc-2338-48af-9478-267403f14dfe None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:42.866695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:42.891258 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17754e81-8395-4221-a2f5-3a22d20849bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:42.892158 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17754e81-8395-4221-a2f5-3a22d20849bb None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:42.892576 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17754e81-8395-4221-a2f5-3a22d20849bb None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:42.892983 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17754e81-8395-4221-a2f5-3a22d20849bb None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:42.894223 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17754e81-8395-4221-a2f5-3a22d20849bb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:42.894731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17754e81-8395-4221-a2f5-3a22d20849bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:42.896659 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17754e81-8395-4221-a2f5-3a22d20849bb None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:42.916923 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 547/1098] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Aug 3 04:55:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:42.923099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17754e81-8395-4221-a2f5-3a22d20849bb None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:42.924189 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315cbc63-7e85-41a8-a85c-f54674c7a8b6 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:42.959909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53aae8-c07b-4779-84d6-92f38eb7158d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:42.960475 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53aae8-c07b-4779-84d6-92f38eb7158d None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:42.960756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53aae8-c07b-4779-84d6-92f38eb7158d None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:42.961029 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53aae8-c07b-4779-84d6-92f38eb7158d None admin] PATH_INFO: `/v3/projects/e1dd3a50e6ca4b0b91e9aadd4fb66d34` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:42.962168 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c53aae8-c07b-4779-84d6-92f38eb7158d None admin] RBAC: Authorizing `identity:get_project(project_id=e1dd3a50e6ca4b0b91e9aadd4fb66d34)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:42.963642 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c53aae8-c07b-4779-84d6-92f38eb7158d None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:42.965320 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 552/1099] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:55:42 2022] GET /identity/v3/projects/e1dd3a50e6ca4b0b91e9aadd4fb66d34 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:42.971256 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53aae8-c07b-4779-84d6-92f38eb7158d None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:43.022631 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17754e81-8395-4221-a2f5-3a22d20849bb None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:43.056003 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545daf1-8e7a-45fc-8548-60ed67a79aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:43.056606 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545daf1-8e7a-45fc-8548-60ed67a79aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:43.056885 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545daf1-8e7a-45fc-8548-60ed67a79aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:43.057158 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545daf1-8e7a-45fc-8548-60ed67a79aa0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:43.069285 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:43.070728 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53aae8-c07b-4779-84d6-92f38eb7158d None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:43.083763 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:43.104408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafae9c0-6733-49cf-8071-c3b2e78e83ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:43.105538 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafae9c0-6733-49cf-8071-c3b2e78e83ed None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:43.105818 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafae9c0-6733-49cf-8071-c3b2e78e83ed None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:43.106082 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafae9c0-6733-49cf-8071-c3b2e78e83ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:43.106868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafae9c0-6733-49cf-8071-c3b2e78e83ed None admin] RBAC: Adding query filter params (role.id=e39d4ecea4c64d5794e1079c9be3ad36, scope.project.id=e1dd3a50e6ca4b0b91e9aadd4fb66d34, user.id=68e8bdea8e0d4b0183d66d2e05056e0b) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:43.107174 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafae9c0-6733-49cf-8071-c3b2e78e83ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:43.108590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafae9c0-6733-49cf-8071-c3b2e78e83ed None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:43.114233 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 553/1100] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:55:42 2022] GET /identity/v3/role_assignments?user.id=68e8bdea8e0d4b0183d66d2e05056e0b&scope.project.id=e1dd3a50e6ca4b0b91e9aadd4fb66d34&role.id=e39d4ecea4c64d5794e1079c9be3ad36 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:43.132159 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545daf1-8e7a-45fc-8548-60ed67a79aa0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:43.133656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545daf1-8e7a-45fc-8548-60ed67a79aa0 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:43.259407 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 548/1101] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:55:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 03 04:55:43.265353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafae9c0-6733-49cf-8071-c3b2e78e83ed None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:43.370002 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafae9c0-6733-49cf-8071-c3b2e78e83ed None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:43.403443 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d108c1b-3e3d-4e70-adcc-e2f8cb5b34d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:43.404080 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d108c1b-3e3d-4e70-adcc-e2f8cb5b34d3 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:43.404362 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d108c1b-3e3d-4e70-adcc-e2f8cb5b34d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:43.404638 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d108c1b-3e3d-4e70-adcc-e2f8cb5b34d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:43.415821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d108c1b-3e3d-4e70-adcc-e2f8cb5b34d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:43.417293 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d108c1b-3e3d-4e70-adcc-e2f8cb5b34d3 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:43.431961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-3d108c1b-3e3d-4e70-adcc-e2f8cb5b34d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:43.431961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:43.431961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:43.431961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:43.431961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:43.431961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:43.431961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:43.431961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:43.431961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:43.431961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:43.431961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:43.431961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:43.431961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:55:43.431961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:55:43.431961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:55:43.432891 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:55:43.432891 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:43.432891 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:43.432891 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:43.432891 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:43.432891 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:43.432891 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:43.432891 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:43.432891 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:43.432891 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:43.432891 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:43.432891 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:43.432891 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:43.432891 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:43.432891 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:43.432891 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:43.432891 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:43.433884 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:55:43.433884 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:55:43.433884 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:55:43.433884 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:43.433884 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 554/1102] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:43.438983 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545daf1-8e7a-45fc-8548-60ed67a79aa0 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:43.482167 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:43.540627 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545daf1-8e7a-45fc-8548-60ed67a79aa0 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:43.575123 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf30910-a449-4790-816c-9c7447c299e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:43.575736 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf30910-a449-4790-816c-9c7447c299e4 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:43.576021 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf30910-a449-4790-816c-9c7447c299e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:43.576300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf30910-a449-4790-816c-9c7447c299e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:43.576985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf30910-a449-4790-816c-9c7447c299e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:43.577293 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf30910-a449-4790-816c-9c7447c299e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:43.578655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf30910-a449-4790-816c-9c7447c299e4 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:43.590419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 549/1103] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:55:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:43.596107 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d108c1b-3e3d-4e70-adcc-e2f8cb5b34d3 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:43.626404 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:43.701434 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d108c1b-3e3d-4e70-adcc-e2f8cb5b34d3 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:43.734808 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c55d84-4783-4555-8046-261149c083b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:43.735621 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c55d84-4783-4555-8046-261149c083b8 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:43.735904 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c55d84-4783-4555-8046-261149c083b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:43.736179 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c55d84-4783-4555-8046-261149c083b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:43.742695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c55d84-4783-4555-8046-261149c083b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:43.744180 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c55d84-4783-4555-8046-261149c083b8 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:43.755356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-36c55d84-4783-4555-8046-261149c083b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:55:43.755356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:55:43.755356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:55:43.755356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:55:43.755356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:55:43.755356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:55:43.755356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:55:43.755356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:55:43.755356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:55:43.755356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:55:43.755356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:55:43.755356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:55:43.755356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:55:43.755356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:55:43.755356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:55:43.756451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:55:43.756451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:55:43.756451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:55:43.756451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:55:43.756451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:55:43.756451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:55:43.756451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:55:43.756451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:55:43.756451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:55:43.756451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:55:43.756451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:55:43.756451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:55:43.756451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:55:43.756451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:55:43.756451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:55:43.756451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:55:43.756451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:55:43.757600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:55:43.757600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:55:43.757600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:55:43.757600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:55:43.757600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:55:43.757600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:55:43.757600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:55:43.757600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:55:43.759129 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 555/1104] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:55:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:55:43.763067 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf30910-a449-4790-816c-9c7447c299e4 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:43.861253 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf30910-a449-4790-816c-9c7447c299e4 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:43.862737 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:43.894833 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531c51f-6589-477c-a9d3-f0f6d19d4bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:43.895471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531c51f-6589-477c-a9d3-f0f6d19d4bff None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:43.895753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531c51f-6589-477c-a9d3-f0f6d19d4bff None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:43.896286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531c51f-6589-477c-a9d3-f0f6d19d4bff None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:43.897069 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531c51f-6589-477c-a9d3-f0f6d19d4bff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:43.897640 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531c51f-6589-477c-a9d3-f0f6d19d4bff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:43.899441 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531c51f-6589-477c-a9d3-f0f6d19d4bff None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:43.912057 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 550/1105] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:55:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:43.918239 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c55d84-4783-4555-8046-261149c083b8 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:44.004977 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:44.017594 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c55d84-4783-4555-8046-261149c083b8 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:44.051394 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034d055-5102-4581-9dff-a7359d5900fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:44.052023 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034d055-5102-4581-9dff-a7359d5900fb None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:44.052306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034d055-5102-4581-9dff-a7359d5900fb None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:44.052581 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034d055-5102-4581-9dff-a7359d5900fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:44.053358 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034d055-5102-4581-9dff-a7359d5900fb None admin] RBAC: Adding query filter params (role.id=729f6ba7aded468788a72fefb5763fc4, scope.project.id=cf3763a37383456184a9ca9351b4ce03, user.id=a3fa4a6c16274c18849869b55b33ac62) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:44.053668 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034d055-5102-4581-9dff-a7359d5900fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:44.055129 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034d055-5102-4581-9dff-a7359d5900fb None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:44.061270 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 556/1106] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:55:43 2022] GET /identity/v3/role_assignments?user.id=a3fa4a6c16274c18849869b55b33ac62&scope.project.id=cf3763a37383456184a9ca9351b4ce03&role.id=729f6ba7aded468788a72fefb5763fc4 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:55:45.163500 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d8b1d-7cde-4a12-836d-31ce2a193ea5 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:45.164048 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d8b1d-7cde-4a12-836d-31ce2a193ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:45.164048 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d8b1d-7cde-4a12-836d-31ce2a193ea5 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:45.165415 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 551/1107] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:45.171266 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2fbf13-7e7a-4b55-9de3-0fdfadbd361b None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:45.171684 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2fbf13-7e7a-4b55-9de3-0fdfadbd361b None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:45.171823 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2fbf13-7e7a-4b55-9de3-0fdfadbd361b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:45.189560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-db2fbf13-7e7a-4b55-9de3-0fdfadbd361b None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:45.190751 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-db2fbf13-7e7a-4b55-9de3-0fdfadbd361b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:45.255935 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 557/1108] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:55:45.281230 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d8b1d-7cde-4a12-836d-31ce2a193ea5 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:45.381008 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d8b1d-7cde-4a12-836d-31ce2a193ea5 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:45.415411 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32addd7f-5882-490a-b8c5-a59f59ec204f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:45.415977 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32addd7f-5882-490a-b8c5-a59f59ec204f None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:45.416261 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32addd7f-5882-490a-b8c5-a59f59ec204f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:45.416540 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32addd7f-5882-490a-b8c5-a59f59ec204f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:45.417802 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32addd7f-5882-490a-b8c5-a59f59ec204f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:45.419289 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32addd7f-5882-490a-b8c5-a59f59ec204f None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:45.420921 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 552/1109] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:45.426597 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2fbf13-7e7a-4b55-9de3-0fdfadbd361b None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:45.505544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:45.523501 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2fbf13-7e7a-4b55-9de3-0fdfadbd361b None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:45.558527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d2ddc-e5be-4f75-8738-df44857e1df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:45.559176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d2ddc-e5be-4f75-8738-df44857e1df6 None admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:45.559458 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d2ddc-e5be-4f75-8738-df44857e1df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:45.559730 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d2ddc-e5be-4f75-8738-df44857e1df6 None admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:45.560420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496d2ddc-e5be-4f75-8738-df44857e1df6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:45.561856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496d2ddc-e5be-4f75-8738-df44857e1df6 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:45.669016 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 558/1110] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 3 04:55:45 2022] POST /identity/v3/users => generated 286 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:55:47.706233 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c9501-a96f-4bdd-8662-a313a795ea2f None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:47.706982 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c9501-a96f-4bdd-8662-a313a795ea2f None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:47.706982 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c9501-a96f-4bdd-8662-a313a795ea2f None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:47.708182 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 553/1111] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:47.713436 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aeed8f-0637-4b3e-87f7-cd3618afe436 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:47.713818 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aeed8f-0637-4b3e-87f7-cd3618afe436 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:47.714544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aeed8f-0637-4b3e-87f7-cd3618afe436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:47.732233 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-06aeed8f-0637-4b3e-87f7-cd3618afe436 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:47.733435 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-06aeed8f-0637-4b3e-87f7-cd3618afe436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:47.798814 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 559/1112] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:55:47.822160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c9501-a96f-4bdd-8662-a313a795ea2f None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:47.852257 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:47.921417 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c9501-a96f-4bdd-8662-a313a795ea2f None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:47.954634 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f5eac-03e3-40b9-b646-9fd45291d8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:47.955351 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f5eac-03e3-40b9-b646-9fd45291d8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:47.955628 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f5eac-03e3-40b9-b646-9fd45291d8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:47.955903 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f5eac-03e3-40b9-b646-9fd45291d8c6 None admin] PATH_INFO: `/v3/roles/561445e71aff40a0af7b01c539d3cfd9` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:47.957326 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f5eac-03e3-40b9-b646-9fd45291d8c6 None admin] RBAC: Authorizing `identity:get_role(role_id=561445e71aff40a0af7b01c539d3cfd9)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:47.958682 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f5eac-03e3-40b9-b646-9fd45291d8c6 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:47.960034 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 554/1113] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:55:47 2022] GET /identity/v3/roles/561445e71aff40a0af7b01c539d3cfd9 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:47.965909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aeed8f-0637-4b3e-87f7-cd3618afe436 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:48.031097 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:48.063101 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aeed8f-0637-4b3e-87f7-cd3618afe436 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:48.096981 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff78487a-2076-4b6e-b4a1-2ed88e23865d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:48.097648 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78487a-2076-4b6e-b4a1-2ed88e23865d None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:48.098065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78487a-2076-4b6e-b4a1-2ed88e23865d None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:48.098766 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78487a-2076-4b6e-b4a1-2ed88e23865d None admin] PATH_INFO: `/v3/users/64814619a1ff4c7e8315682bfa691e2f` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:48.111472 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff78487a-2076-4b6e-b4a1-2ed88e23865d None admin] RBAC: Authorizing `identity:get_user(user_id=64814619a1ff4c7e8315682bfa691e2f)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:48.112905 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff78487a-2076-4b6e-b4a1-2ed88e23865d None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:48.114496 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 560/1114] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:55:47 2022] GET /identity/v3/users/64814619a1ff4c7e8315682bfa691e2f => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:48.120836 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f5eac-03e3-40b9-b646-9fd45291d8c6 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:48.213918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:48.217501 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f5eac-03e3-40b9-b646-9fd45291d8c6 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:48.251098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a4455-61fd-45ce-8774-8c56447c8468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:48.251770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a4455-61fd-45ce-8774-8c56447c8468 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:48.252050 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a4455-61fd-45ce-8774-8c56447c8468 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:48.252321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a4455-61fd-45ce-8774-8c56447c8468 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:48.318240 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721a4455-61fd-45ce-8774-8c56447c8468 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:48.324172 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721a4455-61fd-45ce-8774-8c56447c8468 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:48.414762 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:48.419419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 555/1115] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:55:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 04:55:48.425472 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff78487a-2076-4b6e-b4a1-2ed88e23865d None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:48.564817 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff78487a-2076-4b6e-b4a1-2ed88e23865d None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:48.601815 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:48.612745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd156336-1e25-478d-9688-d9e304dc81f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:48.613389 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd156336-1e25-478d-9688-d9e304dc81f2 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:48.613674 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd156336-1e25-478d-9688-d9e304dc81f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:48.613953 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd156336-1e25-478d-9688-d9e304dc81f2 None admin] PATH_INFO: `/v3/projects/e1dd3a50e6ca4b0b91e9aadd4fb66d34` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:48.615143 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd156336-1e25-478d-9688-d9e304dc81f2 None admin] RBAC: Authorizing `identity:get_project(project_id=e1dd3a50e6ca4b0b91e9aadd4fb66d34)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:48.626369 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd156336-1e25-478d-9688-d9e304dc81f2 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:48.626369 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 561/1116] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:55:48 2022] GET /identity/v3/projects/e1dd3a50e6ca4b0b91e9aadd4fb66d34 => generated 311 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:48.626577 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a4455-61fd-45ce-8774-8c56447c8468 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:48.724258 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a4455-61fd-45ce-8774-8c56447c8468 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:48.757489 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b2c2a-29fe-4eff-bd86-910cdbd8274a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:48.758093 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b2c2a-29fe-4eff-bd86-910cdbd8274a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:48.758376 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b2c2a-29fe-4eff-bd86-910cdbd8274a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:48.758651 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b2c2a-29fe-4eff-bd86-910cdbd8274a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:48.759463 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b2c2a-29fe-4eff-bd86-910cdbd8274a None admin] RBAC: Adding query filter params (role.id=561445e71aff40a0af7b01c539d3cfd9, scope.project.id=e1dd3a50e6ca4b0b91e9aadd4fb66d34, user.id=64814619a1ff4c7e8315682bfa691e2f) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:48.759773 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b2c2a-29fe-4eff-bd86-910cdbd8274a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:48.761244 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b2c2a-29fe-4eff-bd86-910cdbd8274a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:48.766506 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 556/1117] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:55:48 2022] GET /identity/v3/role_assignments?user.id=64814619a1ff4c7e8315682bfa691e2f&scope.project.id=e1dd3a50e6ca4b0b91e9aadd4fb66d34&role.id=561445e71aff40a0af7b01c539d3cfd9 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:50.757763 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0b9e5-c2fe-41f5-bf72-b19acf853558 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:50.758306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0b9e5-c2fe-41f5-bf72-b19acf853558 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:50.758306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0b9e5-c2fe-41f5-bf72-b19acf853558 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:50.759693 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 562/1118] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:50.764854 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a426706-20dd-4dab-8131-8e4b8eb4462d None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:50.765271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a426706-20dd-4dab-8131-8e4b8eb4462d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:50.765408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a426706-20dd-4dab-8131-8e4b8eb4462d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:50.784514 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-6a426706-20dd-4dab-8131-8e4b8eb4462d None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:50.785696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6a426706-20dd-4dab-8131-8e4b8eb4462d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:50.840902 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:50.850913 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 557/1119] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:55:50.873834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0b9e5-c2fe-41f5-bf72-b19acf853558 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:50.975254 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0b9e5-c2fe-41f5-bf72-b19acf853558 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:51.008314 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36fad6-bab3-4763-b20c-29437c9a43f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:51.008944 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36fad6-bab3-4763-b20c-29437c9a43f4 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:51.009225 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36fad6-bab3-4763-b20c-29437c9a43f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:51.009501 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36fad6-bab3-4763-b20c-29437c9a43f4 None admin] PATH_INFO: `/v3/roles/561445e71aff40a0af7b01c539d3cfd9` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:51.010997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36fad6-bab3-4763-b20c-29437c9a43f4 None admin] RBAC: Authorizing `identity:get_role(role_id=561445e71aff40a0af7b01c539d3cfd9)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:51.012361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36fad6-bab3-4763-b20c-29437c9a43f4 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:51.013695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 563/1120] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:55:50 2022] GET /identity/v3/roles/561445e71aff40a0af7b01c539d3cfd9 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:51.019658 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a426706-20dd-4dab-8131-8e4b8eb4462d None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:51.116243 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a426706-20dd-4dab-8131-8e4b8eb4462d None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:51.148665 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e72b9-30cb-4166-afab-605830208042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:51.149290 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e72b9-30cb-4166-afab-605830208042 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:51.149568 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e72b9-30cb-4166-afab-605830208042 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:51.149842 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e72b9-30cb-4166-afab-605830208042 None admin] PATH_INFO: `/v3/users/64814619a1ff4c7e8315682bfa691e2f` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:51.151009 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e72b9-30cb-4166-afab-605830208042 None admin] RBAC: Authorizing `identity:get_user(user_id=64814619a1ff4c7e8315682bfa691e2f)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:51.153066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e72b9-30cb-4166-afab-605830208042 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:51.154637 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 558/1121] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:55:51 2022] GET /identity/v3/users/64814619a1ff4c7e8315682bfa691e2f => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:51.160414 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36fad6-bab3-4763-b20c-29437c9a43f4 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:51.221229 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:51.258707 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36fad6-bab3-4763-b20c-29437c9a43f4 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:51.292719 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ebe735-40b2-48dc-b6d2-889bc90b9562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:51.293342 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebe735-40b2-48dc-b6d2-889bc90b9562 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:51.293619 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebe735-40b2-48dc-b6d2-889bc90b9562 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:51.293889 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebe735-40b2-48dc-b6d2-889bc90b9562 None admin] PATH_INFO: `/v3/projects/e1dd3a50e6ca4b0b91e9aadd4fb66d34` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:51.295048 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ebe735-40b2-48dc-b6d2-889bc90b9562 None admin] RBAC: Authorizing `identity:get_project(project_id=e1dd3a50e6ca4b0b91e9aadd4fb66d34)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:51.296669 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ebe735-40b2-48dc-b6d2-889bc90b9562 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:51.298295 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 564/1122] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:55:51 2022] GET /identity/v3/projects/e1dd3a50e6ca4b0b91e9aadd4fb66d34 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:51.304426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e72b9-30cb-4166-afab-605830208042 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:51.362585 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:51.403053 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e72b9-30cb-4166-afab-605830208042 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:51.436688 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de9007-0073-4c23-b95e-d2bfa157912a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:51.437338 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de9007-0073-4c23-b95e-d2bfa157912a None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:51.437621 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de9007-0073-4c23-b95e-d2bfa157912a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:51.437895 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de9007-0073-4c23-b95e-d2bfa157912a None admin] PATH_INFO: `/v3/projects/e1dd3a50e6ca4b0b91e9aadd4fb66d34/users/64814619a1ff4c7e8315682bfa691e2f/roles/561445e71aff40a0af7b01c539d3cfd9` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:51.440021 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de9007-0073-4c23-b95e-d2bfa157912a None admin] RBAC: Authorizing `identity:create_grant(project_id=e1dd3a50e6ca4b0b91e9aadd4fb66d34, user_id=64814619a1ff4c7e8315682bfa691e2f, role_id=561445e71aff40a0af7b01c539d3cfd9)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:51.441448 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de9007-0073-4c23-b95e-d2bfa157912a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:51.512929 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 559/1123] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 3 04:55:51 2022] PUT /identity/v3/projects/e1dd3a50e6ca4b0b91e9aadd4fb66d34/users/64814619a1ff4c7e8315682bfa691e2f/roles/561445e71aff40a0af7b01c539d3cfd9 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 04:55:53.491395 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79806d35-e8ee-48c8-adcd-273254fe693a None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:53.491926 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79806d35-e8ee-48c8-adcd-273254fe693a None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:53.491926 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79806d35-e8ee-48c8-adcd-273254fe693a None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:53.493266 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 565/1124] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:53.499273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259de418-40ce-497e-b935-81ef0c72d3f2 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:53.499704 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259de418-40ce-497e-b935-81ef0c72d3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:53.499841 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259de418-40ce-497e-b935-81ef0c72d3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:53.518108 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-259de418-40ce-497e-b935-81ef0c72d3f2 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:53.519342 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-259de418-40ce-497e-b935-81ef0c72d3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:53.586267 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 560/1125] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 04:55:53.609795 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79806d35-e8ee-48c8-adcd-273254fe693a None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:53.710391 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79806d35-e8ee-48c8-adcd-273254fe693a None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:53.738918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:53.744449 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc515d6b-37d6-4d0a-9646-977632805b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:53.745102 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc515d6b-37d6-4d0a-9646-977632805b33 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:53.745383 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc515d6b-37d6-4d0a-9646-977632805b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:53.745656 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc515d6b-37d6-4d0a-9646-977632805b33 None admin] PATH_INFO: `/v3/roles/561445e71aff40a0af7b01c539d3cfd9` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:53.747127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc515d6b-37d6-4d0a-9646-977632805b33 None admin] RBAC: Authorizing `identity:get_role(role_id=561445e71aff40a0af7b01c539d3cfd9)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:53.748500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc515d6b-37d6-4d0a-9646-977632805b33 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:53.749816 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 566/1126] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:55:53 2022] GET /identity/v3/roles/561445e71aff40a0af7b01c539d3cfd9 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:53.756002 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259de418-40ce-497e-b935-81ef0c72d3f2 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:53.805400 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:53.853770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259de418-40ce-497e-b935-81ef0c72d3f2 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:53.888128 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d8c5d-53d2-4928-b028-f146c67de26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:53.888762 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d8c5d-53d2-4928-b028-f146c67de26d None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:53.889048 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d8c5d-53d2-4928-b028-f146c67de26d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:53.889328 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d8c5d-53d2-4928-b028-f146c67de26d None admin] PATH_INFO: `/v3/users/64814619a1ff4c7e8315682bfa691e2f` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:53.890485 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935d8c5d-53d2-4928-b028-f146c67de26d None admin] RBAC: Authorizing `identity:get_user(user_id=64814619a1ff4c7e8315682bfa691e2f)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:53.891954 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935d8c5d-53d2-4928-b028-f146c67de26d None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:53.893563 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 561/1127] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:55:53 2022] GET /identity/v3/users/64814619a1ff4c7e8315682bfa691e2f => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:53.899576 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc515d6b-37d6-4d0a-9646-977632805b33 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:53.998584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc515d6b-37d6-4d0a-9646-977632805b33 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:54.032676 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989a277-c727-4be1-a878-576db795b355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:54.033212 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989a277-c727-4be1-a878-576db795b355 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:54.033494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989a277-c727-4be1-a878-576db795b355 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:54.033770 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989a277-c727-4be1-a878-576db795b355 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:54.095773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:54.099208 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5989a277-c727-4be1-a878-576db795b355 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:54.100662 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5989a277-c727-4be1-a878-576db795b355 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:54.196869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 567/1128] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:55:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 04:55:54.203009 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d8c5d-53d2-4928-b028-f146c67de26d None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:54.312702 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d8c5d-53d2-4928-b028-f146c67de26d None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:54.329580 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:54.345740 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89eb9a0-5a58-4c1a-82db-872dd97538ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:54.346378 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89eb9a0-5a58-4c1a-82db-872dd97538ca None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:54.346664 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89eb9a0-5a58-4c1a-82db-872dd97538ca None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:54.347078 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89eb9a0-5a58-4c1a-82db-872dd97538ca None admin] PATH_INFO: `/v3/projects/e1dd3a50e6ca4b0b91e9aadd4fb66d34` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:54.352017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89eb9a0-5a58-4c1a-82db-872dd97538ca None admin] RBAC: Authorizing `identity:get_project(project_id=e1dd3a50e6ca4b0b91e9aadd4fb66d34)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:54.352017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89eb9a0-5a58-4c1a-82db-872dd97538ca None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:54.352017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 562/1129] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:55:54 2022] GET /identity/v3/projects/e1dd3a50e6ca4b0b91e9aadd4fb66d34 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:54.357940 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989a277-c727-4be1-a878-576db795b355 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:54.451293 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:54.455187 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989a277-c727-4be1-a878-576db795b355 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:54.488607 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469a517-2e14-4790-9ec4-7e2cbedad5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:54.489207 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469a517-2e14-4790-9ec4-7e2cbedad5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:54.489485 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469a517-2e14-4790-9ec4-7e2cbedad5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:54.489755 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469a517-2e14-4790-9ec4-7e2cbedad5d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:54.490522 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469a517-2e14-4790-9ec4-7e2cbedad5d1 None admin] RBAC: Adding query filter params (role.id=561445e71aff40a0af7b01c539d3cfd9, scope.project.id=e1dd3a50e6ca4b0b91e9aadd4fb66d34, user.id=64814619a1ff4c7e8315682bfa691e2f) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:55:54.490833 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469a517-2e14-4790-9ec4-7e2cbedad5d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:54.492296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469a517-2e14-4790-9ec4-7e2cbedad5d1 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:54.499299 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 568/1130] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:55:54 2022] GET /identity/v3/role_assignments?user.id=64814619a1ff4c7e8315682bfa691e2f&scope.project.id=e1dd3a50e6ca4b0b91e9aadd4fb66d34&role.id=561445e71aff40a0af7b01c539d3cfd9 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:56.527642 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790a599-6d05-4d7c-bacd-2df0b3033537 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:56.528218 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790a599-6d05-4d7c-bacd-2df0b3033537 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:56.528218 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790a599-6d05-4d7c-bacd-2df0b3033537 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:56.529643 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 563/1131] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:56.535087 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2d20c-5f95-4f70-99c1-22d9b515edc8 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:56.535544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2d20c-5f95-4f70-99c1-22d9b515edc8 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:56.535645 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2d20c-5f95-4f70-99c1-22d9b515edc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:56.558232 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-23b2d20c-5f95-4f70-99c1-22d9b515edc8 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:56.559472 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-23b2d20c-5f95-4f70-99c1-22d9b515edc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:56.635008 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 569/1132] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:55:56.665239 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790a599-6d05-4d7c-bacd-2df0b3033537 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:56.778967 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790a599-6d05-4d7c-bacd-2df0b3033537 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:56.811874 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b33817e-36ed-482b-ad36-56ef9fdc9280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:56.812508 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33817e-36ed-482b-ad36-56ef9fdc9280 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:56.812793 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33817e-36ed-482b-ad36-56ef9fdc9280 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:56.813070 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33817e-36ed-482b-ad36-56ef9fdc9280 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:56.814231 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b33817e-36ed-482b-ad36-56ef9fdc9280 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:56.815702 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b33817e-36ed-482b-ad36-56ef9fdc9280 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:56.817290 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 564/1133] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:56.823254 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2d20c-5f95-4f70-99c1-22d9b515edc8 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:56.911564 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:56.920393 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2d20c-5f95-4f70-99c1-22d9b515edc8 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:56.953936 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40f2a7-2b65-44db-8a98-20ef0992a4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:56.954548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40f2a7-2b65-44db-8a98-20ef0992a4f1 None admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:56.954833 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40f2a7-2b65-44db-8a98-20ef0992a4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:56.955135 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40f2a7-2b65-44db-8a98-20ef0992a4f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:56.955811 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d40f2a7-2b65-44db-8a98-20ef0992a4f1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:56.957245 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d40f2a7-2b65-44db-8a98-20ef0992a4f1 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:57.052708 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 570/1134] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Aug 3 04:55:56 2022] POST /identity/v3/projects => generated 311 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:55:59.120248 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf16c0-76e3-45cd-b85f-d9c0031ff936 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:59.120801 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf16c0-76e3-45cd-b85f-d9c0031ff936 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:59.120801 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf16c0-76e3-45cd-b85f-d9c0031ff936 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:59.122134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 565/1135] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:55:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:55:59.127423 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02baf762-330b-4b04-a7ab-e413593059bc None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:59.127839 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02baf762-330b-4b04-a7ab-e413593059bc None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:59.127985 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02baf762-330b-4b04-a7ab-e413593059bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:59.146145 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-02baf762-330b-4b04-a7ab-e413593059bc None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:55:59.147379 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-02baf762-330b-4b04-a7ab-e413593059bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:55:59.212635 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 571/1136] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:55:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:55:59.238062 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf16c0-76e3-45cd-b85f-d9c0031ff936 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:59.293982 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:59.336197 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf16c0-76e3-45cd-b85f-d9c0031ff936 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:59.370220 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e0ef2-3630-42bd-a685-f2bc455977b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:59.370922 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e0ef2-3630-42bd-a685-f2bc455977b1 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:59.371150 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e0ef2-3630-42bd-a685-f2bc455977b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:59.371425 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e0ef2-3630-42bd-a685-f2bc455977b1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:59.372547 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e0ef2-3630-42bd-a685-f2bc455977b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:59.373968 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e0ef2-3630-42bd-a685-f2bc455977b1 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:59.375576 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 566/1137] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:55:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:55:59.381334 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02baf762-330b-4b04-a7ab-e413593059bc None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:55:59.455087 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:55:59.479337 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02baf762-330b-4b04-a7ab-e413593059bc None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:55:59.512596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0d2787-3392-4d81-9731-ddedd6e2aace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:55:59.513209 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d2787-3392-4d81-9731-ddedd6e2aace None admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:55:59.513494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d2787-3392-4d81-9731-ddedd6e2aace None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:55:59.513774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d2787-3392-4d81-9731-ddedd6e2aace None admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:55:59.514469 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0d2787-3392-4d81-9731-ddedd6e2aace None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:55:59.515948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0d2787-3392-4d81-9731-ddedd6e2aace None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:55:59.615863 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 572/1138] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 3 04:55:59 2022] POST /identity/v3/users => generated 286 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:56:01.662886 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878464d0-37ee-4948-be21-874c4c2b09b3 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:01.663447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878464d0-37ee-4948-be21-874c4c2b09b3 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:01.663447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878464d0-37ee-4948-be21-874c4c2b09b3 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:01.664775 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 567/1139] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:56:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:56:01.670612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f132f9c-2113-4a18-9c4f-ae837d43164b None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:01.671065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f132f9c-2113-4a18-9c4f-ae837d43164b None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:01.671198 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f132f9c-2113-4a18-9c4f-ae837d43164b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:01.689610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-7f132f9c-2113-4a18-9c4f-ae837d43164b None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:56:01.690811 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-7f132f9c-2113-4a18-9c4f-ae837d43164b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:56:01.756666 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 573/1140] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:56:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:56:01.779649 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878464d0-37ee-4948-be21-874c4c2b09b3 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:01.881536 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878464d0-37ee-4948-be21-874c4c2b09b3 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:01.903559 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:01.921410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233cf32e-277b-43a2-a850-43c1d0b03e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:01.921410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233cf32e-277b-43a2-a850-43c1d0b03e2b None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:01.921410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233cf32e-277b-43a2-a850-43c1d0b03e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:01.921410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233cf32e-277b-43a2-a850-43c1d0b03e2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:01.926151 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233cf32e-277b-43a2-a850-43c1d0b03e2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:56:01.931058 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-233cf32e-277b-43a2-a850-43c1d0b03e2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:56:01.931058 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:56:01.931058 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:56:01.931058 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:56:01.931058 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:56:01.931058 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:56:01.931058 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:56:01.931058 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:56:01.931058 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:56:01.931058 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:56:01.931058 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:56:01.931058 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:56:01.931058 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:56:01.931058 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:56:01.931058 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:56:01.931058 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:56:01.932807 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:56:01.932807 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:56:01.932807 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:56:01.932807 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:56:01.932807 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:56:01.932807 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:56:01.932807 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:56:01.932807 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:56:01.932807 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:56:01.932807 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:56:01.932807 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:56:01.932807 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:56:01.932807 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:56:01.932807 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:56:01.932807 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:56:01.932807 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:56:01.934132 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:56:01.934132 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:56:01.934132 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:56:01.934132 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:56:01.934132 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:56:01.934132 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:56:01.934132 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:56:01.934132 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:56:01.934132 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:56:01.934132 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:56:01.934132 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:56:01.934132 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:56:01.934132 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 568/1141] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:56:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:56:01.938858 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f132f9c-2113-4a18-9c4f-ae837d43164b None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:01.996521 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:02.036664 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f132f9c-2113-4a18-9c4f-ae837d43164b None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:02.070659 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c75414c-9f9d-44c2-a2af-f0c8a7eb1510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:02.071303 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c75414c-9f9d-44c2-a2af-f0c8a7eb1510 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:02.071585 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c75414c-9f9d-44c2-a2af-f0c8a7eb1510 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:02.071862 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c75414c-9f9d-44c2-a2af-f0c8a7eb1510 None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:02.072570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c75414c-9f9d-44c2-a2af-f0c8a7eb1510 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:56:02.072873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c75414c-9f9d-44c2-a2af-f0c8a7eb1510 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:02.074245 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c75414c-9f9d-44c2-a2af-f0c8a7eb1510 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:02.082193 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 574/1142] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:56:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:56:02.088061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233cf32e-277b-43a2-a850-43c1d0b03e2b None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:02.186495 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233cf32e-277b-43a2-a850-43c1d0b03e2b None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:02.219395 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a9ca5-1b0d-459d-b4b4-e74690f650a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:02.220014 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a9ca5-1b0d-459d-b4b4-e74690f650a5 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:02.220296 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a9ca5-1b0d-459d-b4b4-e74690f650a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:02.220571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a9ca5-1b0d-459d-b4b4-e74690f650a5 None admin] PATH_INFO: `/v3/users/6fd77a62171748dcaf2e2c27f705cc3b` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:02.233810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a9ca5-1b0d-459d-b4b4-e74690f650a5 None admin] RBAC: Authorizing `identity:get_user(user_id=6fd77a62171748dcaf2e2c27f705cc3b)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:02.235276 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a9ca5-1b0d-459d-b4b4-e74690f650a5 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:02.236915 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 569/1143] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:56:02 2022] GET /identity/v3/users/6fd77a62171748dcaf2e2c27f705cc3b => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:56:02.242665 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c75414c-9f9d-44c2-a2af-f0c8a7eb1510 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:02.347185 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c75414c-9f9d-44c2-a2af-f0c8a7eb1510 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:02.369887 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:02.381437 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e62dc-d68a-4995-b280-3e423cd8cd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:02.382055 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e62dc-d68a-4995-b280-3e423cd8cd2c None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:02.382339 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e62dc-d68a-4995-b280-3e423cd8cd2c None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:02.382617 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e62dc-d68a-4995-b280-3e423cd8cd2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:02.450699 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e62dc-d68a-4995-b280-3e423cd8cd2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:02.452114 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e62dc-d68a-4995-b280-3e423cd8cd2c None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:02.547636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 575/1144] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:56:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 04:56:02.553685 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a9ca5-1b0d-459d-b4b4-e74690f650a5 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:02.594896 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:02.651384 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a9ca5-1b0d-459d-b4b4-e74690f650a5 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:02.702033 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b7835-69f8-49bf-9404-fc6eb82edad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:02.702684 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b7835-69f8-49bf-9404-fc6eb82edad9 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:02.702887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b7835-69f8-49bf-9404-fc6eb82edad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:02.703170 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b7835-69f8-49bf-9404-fc6eb82edad9 None admin] PATH_INFO: `/v3/projects/0c632061c4034b8a84a71ec750b05d15` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:02.704321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b7835-69f8-49bf-9404-fc6eb82edad9 None admin] RBAC: Authorizing `identity:get_project(project_id=0c632061c4034b8a84a71ec750b05d15)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:02.705755 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b7835-69f8-49bf-9404-fc6eb82edad9 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:02.707494 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 570/1145] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:56:02 2022] GET /identity/v3/projects/0c632061c4034b8a84a71ec750b05d15 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:56:02.714424 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e62dc-d68a-4995-b280-3e423cd8cd2c None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:02.737856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:02.816709 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e62dc-d68a-4995-b280-3e423cd8cd2c None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:02.851649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22adf86-fea4-4a54-b0ab-d3dba4d14142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:02.852261 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22adf86-fea4-4a54-b0ab-d3dba4d14142 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:02.852539 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22adf86-fea4-4a54-b0ab-d3dba4d14142 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:02.852812 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22adf86-fea4-4a54-b0ab-d3dba4d14142 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:02.853584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22adf86-fea4-4a54-b0ab-d3dba4d14142 None admin] RBAC: Adding query filter params (role.id=e39d4ecea4c64d5794e1079c9be3ad36, scope.project.id=0c632061c4034b8a84a71ec750b05d15, user.id=6fd77a62171748dcaf2e2c27f705cc3b) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:56:02.853891 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22adf86-fea4-4a54-b0ab-d3dba4d14142 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:02.855352 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22adf86-fea4-4a54-b0ab-d3dba4d14142 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:02.860587 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 576/1146] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:56:02 2022] GET /identity/v3/role_assignments?user.id=6fd77a62171748dcaf2e2c27f705cc3b&scope.project.id=0c632061c4034b8a84a71ec750b05d15&role.id=e39d4ecea4c64d5794e1079c9be3ad36 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:56:04.834945 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469025a-82b2-4b85-abb9-eb8045978103 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:04.835493 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469025a-82b2-4b85-abb9-eb8045978103 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:04.835493 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469025a-82b2-4b85-abb9-eb8045978103 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:04.836850 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 571/1147] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:56:04.842647 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75814a-d36a-4ab2-88ae-ec0f0c475681 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:04.843123 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75814a-d36a-4ab2-88ae-ec0f0c475681 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:04.843214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75814a-d36a-4ab2-88ae-ec0f0c475681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:04.861988 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-6e75814a-d36a-4ab2-88ae-ec0f0c475681 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:56:04.863222 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-6e75814a-d36a-4ab2-88ae-ec0f0c475681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:56:04.928452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 577/1148] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:56:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:56:04.950714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1469025a-82b2-4b85-abb9-eb8045978103 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:05.051449 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1469025a-82b2-4b85-abb9-eb8045978103 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:05.060829 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:05.084516 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2987cfcc-1ff6-4886-95f4-250a2b1107d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:05.085137 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987cfcc-1ff6-4886-95f4-250a2b1107d5 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:05.085416 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987cfcc-1ff6-4886-95f4-250a2b1107d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:05.085688 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987cfcc-1ff6-4886-95f4-250a2b1107d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:05.096486 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2987cfcc-1ff6-4886-95f4-250a2b1107d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:56:05.101784 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-2987cfcc-1ff6-4886-95f4-250a2b1107d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:56:05.101784 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:56:05.101784 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:56:05.101784 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:56:05.101784 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:56:05.101784 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:56:05.101784 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:56:05.101784 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:56:05.101784 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:56:05.101784 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:56:05.101784 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:56:05.101784 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:56:05.101784 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:56:05.101784 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:56:05.101784 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:56:05.101784 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:56:05.102956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:56:05.102956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:56:05.102956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:56:05.102956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:56:05.102956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:56:05.102956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:56:05.102956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:56:05.102956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:56:05.102956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:56:05.102956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:56:05.102956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:56:05.102956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:56:05.102956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:56:05.102956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:56:05.102956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:56:05.102956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:56:05.104044 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:56:05.104044 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:56:05.104044 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:56:05.104044 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:56:05.104044 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:56:05.104044 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:56:05.104044 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:56:05.104044 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:56:05.104044 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:56:05.104044 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:56:05.104044 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:56:05.104044 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:56:05.104044 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 572/1149] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:56:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:56:05.109004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75814a-d36a-4ab2-88ae-ec0f0c475681 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:05.126615 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:05.206670 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75814a-d36a-4ab2-88ae-ec0f0c475681 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:05.239087 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214557ee-fb9a-4612-9156-74a0b92d7a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:05.239698 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214557ee-fb9a-4612-9156-74a0b92d7a73 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:05.239983 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214557ee-fb9a-4612-9156-74a0b92d7a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:05.240258 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214557ee-fb9a-4612-9156-74a0b92d7a73 None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:05.240969 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214557ee-fb9a-4612-9156-74a0b92d7a73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:56:05.241275 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214557ee-fb9a-4612-9156-74a0b92d7a73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:05.242646 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214557ee-fb9a-4612-9156-74a0b92d7a73 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:05.251242 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 578/1150] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:56:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:56:05.257104 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2987cfcc-1ff6-4886-95f4-250a2b1107d5 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:05.376510 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2987cfcc-1ff6-4886-95f4-250a2b1107d5 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:05.412974 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2836065-ac4c-4c21-ba3f-0a0d6fcff000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:05.413608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2836065-ac4c-4c21-ba3f-0a0d6fcff000 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:05.413892 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2836065-ac4c-4c21-ba3f-0a0d6fcff000 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:05.414163 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2836065-ac4c-4c21-ba3f-0a0d6fcff000 None admin] PATH_INFO: `/v3/users/6fd77a62171748dcaf2e2c27f705cc3b` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:05.415337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2836065-ac4c-4c21-ba3f-0a0d6fcff000 None admin] RBAC: Authorizing `identity:get_user(user_id=6fd77a62171748dcaf2e2c27f705cc3b)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:05.417319 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2836065-ac4c-4c21-ba3f-0a0d6fcff000 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:05.418931 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 573/1151] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:56:05 2022] GET /identity/v3/users/6fd77a62171748dcaf2e2c27f705cc3b => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:56:05.424733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214557ee-fb9a-4612-9156-74a0b92d7a73 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:05.502060 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:05.523069 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214557ee-fb9a-4612-9156-74a0b92d7a73 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:05.557548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b390b92e-e03c-4f13-a2e0-a3d79ef1f36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:05.558180 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390b92e-e03c-4f13-a2e0-a3d79ef1f36e None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:05.558464 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390b92e-e03c-4f13-a2e0-a3d79ef1f36e None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:05.558745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390b92e-e03c-4f13-a2e0-a3d79ef1f36e None admin] PATH_INFO: `/v3/projects/0c632061c4034b8a84a71ec750b05d15` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:05.559978 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b390b92e-e03c-4f13-a2e0-a3d79ef1f36e None admin] RBAC: Authorizing `identity:get_project(project_id=0c632061c4034b8a84a71ec750b05d15)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:05.561416 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b390b92e-e03c-4f13-a2e0-a3d79ef1f36e None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:05.563100 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 579/1152] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:56:05 2022] GET /identity/v3/projects/0c632061c4034b8a84a71ec750b05d15 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:56:05.569029 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2836065-ac4c-4c21-ba3f-0a0d6fcff000 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:05.603171 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:05.668574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2836065-ac4c-4c21-ba3f-0a0d6fcff000 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:05.713922 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a8c81a-2489-488b-a677-34103976666c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:05.714579 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8c81a-2489-488b-a677-34103976666c None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:05.714877 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8c81a-2489-488b-a677-34103976666c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:05.715159 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8c81a-2489-488b-a677-34103976666c None admin] PATH_INFO: `/v3/projects/0c632061c4034b8a84a71ec750b05d15/users/6fd77a62171748dcaf2e2c27f705cc3b/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:05.717057 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a8c81a-2489-488b-a677-34103976666c None admin] RBAC: Authorizing `identity:create_grant(project_id=0c632061c4034b8a84a71ec750b05d15, user_id=6fd77a62171748dcaf2e2c27f705cc3b, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:05.718993 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a8c81a-2489-488b-a677-34103976666c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:05.787893 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 574/1153] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 3 04:56:05 2022] PUT /identity/v3/projects/0c632061c4034b8a84a71ec750b05d15/users/6fd77a62171748dcaf2e2c27f705cc3b/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 04:56:07.801307 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10281719-3044-4f45-8629-b2fd2f71b123 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:07.801870 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10281719-3044-4f45-8629-b2fd2f71b123 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:07.801870 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10281719-3044-4f45-8629-b2fd2f71b123 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:07.803240 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 580/1154] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:56:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:56:07.808274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc79d9f-662d-49f3-970c-8aa8076ce188 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:07.808681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc79d9f-662d-49f3-970c-8aa8076ce188 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:07.808821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc79d9f-662d-49f3-970c-8aa8076ce188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:07.827077 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-cfc79d9f-662d-49f3-970c-8aa8076ce188 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:56:07.828241 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-cfc79d9f-662d-49f3-970c-8aa8076ce188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:56:07.892600 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 575/1155] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:56:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:56:07.917443 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10281719-3044-4f45-8629-b2fd2f71b123 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:08.017126 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10281719-3044-4f45-8629-b2fd2f71b123 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:08.050632 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d78af0-affb-4ef4-a269-8aaff445829e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:08.051298 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d78af0-affb-4ef4-a269-8aaff445829e None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:08.051585 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d78af0-affb-4ef4-a269-8aaff445829e None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:08.051862 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d78af0-affb-4ef4-a269-8aaff445829e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:08.062862 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d78af0-affb-4ef4-a269-8aaff445829e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:56:08.067742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-88d78af0-affb-4ef4-a269-8aaff445829e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:56:08.067742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:56:08.067742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:56:08.067742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:56:08.067742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:56:08.067742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:56:08.067742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:56:08.067742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:56:08.067742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:56:08.067742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:56:08.067742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:56:08.067742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:56:08.067742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:56:08.067742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:56:08.067742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:56:08.067742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:56:08.068845 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:56:08.068845 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:56:08.068845 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:56:08.068845 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:56:08.068845 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:56:08.068845 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:56:08.068845 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:56:08.068845 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:56:08.068845 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:56:08.068845 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:56:08.068845 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:56:08.068845 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:56:08.068845 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:56:08.068845 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:56:08.068845 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:56:08.068845 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:56:08.073311 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:56:08.073311 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:56:08.073311 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:56:08.073311 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:56:08.073311 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:56:08.073311 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:56:08.073311 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:56:08.073311 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:56:08.073311 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:56:08.073311 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:56:08.073311 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:56:08.073311 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:56:08.073311 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 581/1156] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:56:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:56:08.075564 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc79d9f-662d-49f3-970c-8aa8076ce188 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:08.101823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:08.172525 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc79d9f-662d-49f3-970c-8aa8076ce188 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:08.204935 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1172333-f2c2-4a10-af83-44adb49c8060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:08.205547 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1172333-f2c2-4a10-af83-44adb49c8060 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:08.205832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1172333-f2c2-4a10-af83-44adb49c8060 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:08.206108 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1172333-f2c2-4a10-af83-44adb49c8060 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:08.206817 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1172333-f2c2-4a10-af83-44adb49c8060 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:56:08.207179 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1172333-f2c2-4a10-af83-44adb49c8060 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:08.208541 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1172333-f2c2-4a10-af83-44adb49c8060 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:08.216904 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 576/1157] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:56:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:56:08.222863 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d78af0-affb-4ef4-a269-8aaff445829e None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:08.232523 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:08.321325 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d78af0-affb-4ef4-a269-8aaff445829e None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:08.355701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d075243-9637-44ae-b7fc-98f62d3506f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:08.356341 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d075243-9637-44ae-b7fc-98f62d3506f5 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:08.359250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d075243-9637-44ae-b7fc-98f62d3506f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:08.359250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d075243-9637-44ae-b7fc-98f62d3506f5 None admin] PATH_INFO: `/v3/users/6fd77a62171748dcaf2e2c27f705cc3b` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:08.359250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d075243-9637-44ae-b7fc-98f62d3506f5 None admin] RBAC: Authorizing `identity:get_user(user_id=6fd77a62171748dcaf2e2c27f705cc3b)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:08.359619 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d075243-9637-44ae-b7fc-98f62d3506f5 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:08.361214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 582/1158] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:56:08 2022] GET /identity/v3/users/6fd77a62171748dcaf2e2c27f705cc3b => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:56:08.367062 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1172333-f2c2-4a10-af83-44adb49c8060 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:08.458902 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:08.471362 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1172333-f2c2-4a10-af83-44adb49c8060 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:08.503740 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44f9ef-d319-41e7-afe4-1de93611cf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:08.504340 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44f9ef-d319-41e7-afe4-1de93611cf24 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:08.504630 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44f9ef-d319-41e7-afe4-1de93611cf24 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:08.504907 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44f9ef-d319-41e7-afe4-1de93611cf24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:08.571466 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df44f9ef-d319-41e7-afe4-1de93611cf24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:08.573414 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df44f9ef-d319-41e7-afe4-1de93611cf24 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:08.657865 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:08.670337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 577/1159] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:56:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 04:56:08.676219 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d075243-9637-44ae-b7fc-98f62d3506f5 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:08.754511 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:08.779516 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d075243-9637-44ae-b7fc-98f62d3506f5 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:08.821892 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647abe9-9ce5-4189-8147-6dbce7a458eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:08.822563 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647abe9-9ce5-4189-8147-6dbce7a458eb None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:08.822805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647abe9-9ce5-4189-8147-6dbce7a458eb None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:08.823115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647abe9-9ce5-4189-8147-6dbce7a458eb None admin] PATH_INFO: `/v3/projects/0c632061c4034b8a84a71ec750b05d15` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:08.824277 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5647abe9-9ce5-4189-8147-6dbce7a458eb None admin] RBAC: Authorizing `identity:get_project(project_id=0c632061c4034b8a84a71ec750b05d15)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:08.825713 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5647abe9-9ce5-4189-8147-6dbce7a458eb None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:08.827413 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 583/1160] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:56:08 2022] GET /identity/v3/projects/0c632061c4034b8a84a71ec750b05d15 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:56:08.833655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44f9ef-d319-41e7-afe4-1de93611cf24 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:08.932099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44f9ef-d319-41e7-afe4-1de93611cf24 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:08.965729 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9dcd0-dc2b-4abd-88f2-2977c5c4803b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:08.966334 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9dcd0-dc2b-4abd-88f2-2977c5c4803b None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:08.966620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9dcd0-dc2b-4abd-88f2-2977c5c4803b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:08.966927 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9dcd0-dc2b-4abd-88f2-2977c5c4803b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:08.967712 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9dcd0-dc2b-4abd-88f2-2977c5c4803b None admin] RBAC: Adding query filter params (role.id=e39d4ecea4c64d5794e1079c9be3ad36, scope.project.id=0c632061c4034b8a84a71ec750b05d15, user.id=6fd77a62171748dcaf2e2c27f705cc3b) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:56:08.968024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9dcd0-dc2b-4abd-88f2-2977c5c4803b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:08.969460 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9dcd0-dc2b-4abd-88f2-2977c5c4803b None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:08.975183 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 578/1161] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:56:08 2022] GET /identity/v3/role_assignments?user.id=6fd77a62171748dcaf2e2c27f705cc3b&scope.project.id=0c632061c4034b8a84a71ec750b05d15&role.id=e39d4ecea4c64d5794e1079c9be3ad36 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:56:10.999053 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa912528-93a3-42ff-a264-4908a0dbe4fb None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:10.999559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa912528-93a3-42ff-a264-4908a0dbe4fb None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:10.999652 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa912528-93a3-42ff-a264-4908a0dbe4fb None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:11.000936 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 584/1162] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:56:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:56:11.006049 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215deca-9308-4ea5-9d07-e7de2981d99f None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:11.006453 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215deca-9308-4ea5-9d07-e7de2981d99f None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:11.006601 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215deca-9308-4ea5-9d07-e7de2981d99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:11.025687 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-c215deca-9308-4ea5-9d07-e7de2981d99f None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:56:11.026884 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c215deca-9308-4ea5-9d07-e7de2981d99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:56:11.072662 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:11.091542 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 579/1163] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:56:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:56:11.113939 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa912528-93a3-42ff-a264-4908a0dbe4fb None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:11.215329 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa912528-93a3-42ff-a264-4908a0dbe4fb None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:11.247921 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551b0d5-0017-4d03-b8e0-64961768ab1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:11.248531 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551b0d5-0017-4d03-b8e0-64961768ab1b None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:11.248814 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551b0d5-0017-4d03-b8e0-64961768ab1b None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:11.249140 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551b0d5-0017-4d03-b8e0-64961768ab1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:11.261121 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:11.316437 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551b0d5-0017-4d03-b8e0-64961768ab1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:11.317899 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551b0d5-0017-4d03-b8e0-64961768ab1b None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:11.413036 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 585/1164] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:56:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 04:56:11.419660 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215deca-9308-4ea5-9d07-e7de2981d99f None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:11.515811 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215deca-9308-4ea5-9d07-e7de2981d99f None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:11.549105 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e6e37-a5f0-4e06-b1b9-f943da3df8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:11.549735 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e6e37-a5f0-4e06-b1b9-f943da3df8f8 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:11.550020 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e6e37-a5f0-4e06-b1b9-f943da3df8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:11.550299 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e6e37-a5f0-4e06-b1b9-f943da3df8f8 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:11.556887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e6e37-a5f0-4e06-b1b9-f943da3df8f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:11.558333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e6e37-a5f0-4e06-b1b9-f943da3df8f8 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:11.569562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-779e6e37-a5f0-4e06-b1b9-f943da3df8f8 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 03 04:56:11.569562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:56:11.569562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 03 04:56:11.569562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 03 04:56:11.569562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:56:11.569562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:56:11.569562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:56:11.569562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:56:11.569562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 03 04:56:11.569562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:56:11.569562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:56:11.569562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:56:11.569562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:56:11.569562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:56:11.569562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:56:11.569562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:56:11.569562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:56:11.570887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:56:11.570887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:56:11.570887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:56:11.570887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:56:11.570887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:56:11.570887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:56:11.570887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:56:11.570887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:56:11.570887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:56:11.570887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:56:11.570887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:56:11.570887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:56:11.570887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:56:11.570887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:56:11.570887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:56:11.570887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:56:11.571782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:56:11.571782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:56:11.571782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:56:11.571782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:56:11.571782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:56:11.571782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:56:11.571782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:56:11.571782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:56:11.571782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:56:11.571782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:56:11.571782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 03 04:56:11.571782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:56:11.571782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 03 04:56:11.571782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:56:11.571782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 580/1165] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Aug 3 04:56:11 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:56:11.577228 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551b0d5-0017-4d03-b8e0-64961768ab1b None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:11.619266 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:11.675814 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551b0d5-0017-4d03-b8e0-64961768ab1b None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:11.710357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d33e44-308d-4d2a-86b4-a981b6b653f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:11.711007 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d33e44-308d-4d2a-86b4-a981b6b653f5 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:11.711292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d33e44-308d-4d2a-86b4-a981b6b653f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:11.711570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d33e44-308d-4d2a-86b4-a981b6b653f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:11.712258 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d33e44-308d-4d2a-86b4-a981b6b653f5 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:56:11.713113 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d33e44-308d-4d2a-86b4-a981b6b653f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:11.714499 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d33e44-308d-4d2a-86b4-a981b6b653f5 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:11.721557 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 586/1166] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Aug 3 04:56:11 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:56:13.679970 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a1e00-c3fe-4720-8794-79a89045f4f8 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:13.680413 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a1e00-c3fe-4720-8794-79a89045f4f8 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:13.680518 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a1e00-c3fe-4720-8794-79a89045f4f8 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:13.681910 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 581/1167] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:56:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:56:13.687209 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed38a38-6e97-4e2b-8e6b-927c03edf882 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:13.687612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed38a38-6e97-4e2b-8e6b-927c03edf882 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:13.687844 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed38a38-6e97-4e2b-8e6b-927c03edf882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:13.706023 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-eed38a38-6e97-4e2b-8e6b-927c03edf882 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:56:13.707257 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-eed38a38-6e97-4e2b-8e6b-927c03edf882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:56:13.774556 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 587/1168] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:56:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:56:13.797742 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a1e00-c3fe-4720-8794-79a89045f4f8 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:13.842615 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:13.900650 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a1e00-c3fe-4720-8794-79a89045f4f8 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:13.934661 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3482d98-3513-4c33-a318-7a963e86f318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:13.935228 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3482d98-3513-4c33-a318-7a963e86f318 None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:13.935518 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3482d98-3513-4c33-a318-7a963e86f318 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:13.935797 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3482d98-3513-4c33-a318-7a963e86f318 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:13.936508 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3482d98-3513-4c33-a318-7a963e86f318 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:13.937912 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3482d98-3513-4c33-a318-7a963e86f318 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:13.993006 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 582/1169] 10.222.0.40 () {60 vars in 1138 bytes} [Wed Aug 3 04:56:13 2022] POST /identity/v3/domains => generated 267 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:56:16.027094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7093ea3-7bcd-46e8-bf78-2b1334573927 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:16.027601 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7093ea3-7bcd-46e8-bf78-2b1334573927 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:16.027720 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7093ea3-7bcd-46e8-bf78-2b1334573927 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:16.028969 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 588/1170] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:56:16.034176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73caac-e8c4-43a9-a921-e6008cc5e37b None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:16.034558 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73caac-e8c4-43a9-a921-e6008cc5e37b None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:16.034731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73caac-e8c4-43a9-a921-e6008cc5e37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:16.052913 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-3e73caac-e8c4-43a9-a921-e6008cc5e37b None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:56:16.054085 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3e73caac-e8c4-43a9-a921-e6008cc5e37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:56:16.118382 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 583/1171] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:56:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:56:16.141879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7093ea3-7bcd-46e8-bf78-2b1334573927 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:16.178070 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:16.243140 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7093ea3-7bcd-46e8-bf78-2b1334573927 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:16.278552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec30915-f796-47cf-88a1-da93d8fe7837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:16.279142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec30915-f796-47cf-88a1-da93d8fe7837 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:16.279428 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec30915-f796-47cf-88a1-da93d8fe7837 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:16.279702 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec30915-f796-47cf-88a1-da93d8fe7837 None admin] PATH_INFO: `/v3/domains/6b934c894582490485408d59d2713b55` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:16.289973 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec30915-f796-47cf-88a1-da93d8fe7837 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6b934c894582490485408d59d2713b55)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:16.291494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec30915-f796-47cf-88a1-da93d8fe7837 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:16.293091 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 589/1172] 10.222.0.40 () {56 vars in 1183 bytes} [Wed Aug 3 04:56:16 2022] GET /identity/v3/domains/6b934c894582490485408d59d2713b55 => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:56:16.298970 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e73caac-e8c4-43a9-a921-e6008cc5e37b None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:16.329678 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:16.397699 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e73caac-e8c4-43a9-a921-e6008cc5e37b None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:16.430731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c071bd-b27e-4dcc-99f1-f65b111ec765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:16.431898 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c071bd-b27e-4dcc-99f1-f65b111ec765 None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:16.432184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c071bd-b27e-4dcc-99f1-f65b111ec765 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:16.432460 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c071bd-b27e-4dcc-99f1-f65b111ec765 None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:16.433153 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c071bd-b27e-4dcc-99f1-f65b111ec765 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:16.434583 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c071bd-b27e-4dcc-99f1-f65b111ec765 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:16.508235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 584/1173] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Aug 3 04:56:16 2022] POST /identity/v3/projects => generated 361 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:56:18.534899 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ee533-b58b-4a76-a020-8579375e281e None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:18.535429 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ee533-b58b-4a76-a020-8579375e281e None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:18.535429 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ee533-b58b-4a76-a020-8579375e281e None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:18.536803 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 590/1174] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:56:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:56:18.542512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e255a-341c-407c-9b7f-8f27420e68ad None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:18.542928 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e255a-341c-407c-9b7f-8f27420e68ad None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:18.543113 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e255a-341c-407c-9b7f-8f27420e68ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:18.561178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-602e255a-341c-407c-9b7f-8f27420e68ad None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:56:18.562368 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-602e255a-341c-407c-9b7f-8f27420e68ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:56:18.627611 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 585/1175] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:56:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:56:18.652798 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ee533-b58b-4a76-a020-8579375e281e None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:18.746744 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:18.753294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ee533-b58b-4a76-a020-8579375e281e None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:18.786420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c942f17-d644-4249-80ec-56bd297626e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:18.787020 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c942f17-d644-4249-80ec-56bd297626e5 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:18.787539 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c942f17-d644-4249-80ec-56bd297626e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:18.788016 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c942f17-d644-4249-80ec-56bd297626e5 None admin] PATH_INFO: `/v3/domains/6b934c894582490485408d59d2713b55` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:18.789530 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c942f17-d644-4249-80ec-56bd297626e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6b934c894582490485408d59d2713b55)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:18.791007 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c942f17-d644-4249-80ec-56bd297626e5 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:18.792589 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 591/1176] 10.222.0.40 () {56 vars in 1183 bytes} [Wed Aug 3 04:56:18 2022] GET /identity/v3/domains/6b934c894582490485408d59d2713b55 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:56:18.798494 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e255a-341c-407c-9b7f-8f27420e68ad None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:18.813919 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:18.896825 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e255a-341c-407c-9b7f-8f27420e68ad None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:18.934457 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d853f3-3b20-429f-8b9f-033ce193c206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:18.935107 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d853f3-3b20-429f-8b9f-033ce193c206 None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:18.935395 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d853f3-3b20-429f-8b9f-033ce193c206 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:18.935674 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d853f3-3b20-429f-8b9f-033ce193c206 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:18.936359 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d853f3-3b20-429f-8b9f-033ce193c206 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:18.937800 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d853f3-3b20-429f-8b9f-033ce193c206 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:19.021739 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 586/1177] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 3 04:56:18 2022] POST /identity/v3/users => generated 311 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:56:21.059382 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d124ef-467d-4568-bd6a-4af43ff73288 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:21.059944 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d124ef-467d-4568-bd6a-4af43ff73288 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:21.059944 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d124ef-467d-4568-bd6a-4af43ff73288 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:21.061268 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 592/1178] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:56:21.066298 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502429a1-ea98-4a7d-b5ac-178b960ba6d3 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:21.066681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502429a1-ea98-4a7d-b5ac-178b960ba6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:21.066874 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502429a1-ea98-4a7d-b5ac-178b960ba6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:21.085173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-502429a1-ea98-4a7d-b5ac-178b960ba6d3 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:56:21.086365 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-502429a1-ea98-4a7d-b5ac-178b960ba6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:56:21.148435 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:21.153092 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 587/1179] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:56:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:56:21.186886 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d124ef-467d-4568-bd6a-4af43ff73288 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:21.294307 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d124ef-467d-4568-bd6a-4af43ff73288 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:21.327735 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb469dc-b19c-4660-ab99-e84266bb2a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:21.328291 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb469dc-b19c-4660-ab99-e84266bb2a68 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:21.328571 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb469dc-b19c-4660-ab99-e84266bb2a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:21.328844 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb469dc-b19c-4660-ab99-e84266bb2a68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:21.339772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb469dc-b19c-4660-ab99-e84266bb2a68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:56:21.344552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-acb469dc-b19c-4660-ab99-e84266bb2a68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:56:21.344552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:56:21.344552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:56:21.344552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:56:21.344552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:56:21.344552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:56:21.344552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:56:21.344552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:56:21.344552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:56:21.344552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:56:21.344552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:56:21.344552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:56:21.344552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:56:21.344552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:56:21.344552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:56:21.344552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:56:21.347630 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:56:21.347630 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:56:21.347630 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:56:21.347630 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:56:21.347630 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:56:21.347630 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:56:21.347630 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:56:21.347630 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:56:21.347630 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:56:21.347630 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:56:21.347630 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:56:21.347630 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:56:21.347630 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:56:21.347630 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:56:21.347630 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:56:21.347630 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:56:21.348687 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:56:21.348687 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:56:21.348687 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:56:21.348687 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:56:21.348687 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:56:21.348687 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:56:21.348687 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:56:21.348687 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:56:21.348687 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:56:21.348687 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:56:21.348687 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:56:21.348687 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:56:21.348687 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 593/1180] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:56:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:56:21.351939 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502429a1-ea98-4a7d-b5ac-178b960ba6d3 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:21.450603 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502429a1-ea98-4a7d-b5ac-178b960ba6d3 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:21.484168 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fef854-0be0-4850-ae16-2fbbf4af32f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:21.484775 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fef854-0be0-4850-ae16-2fbbf4af32f3 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:21.485059 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fef854-0be0-4850-ae16-2fbbf4af32f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:21.485335 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fef854-0be0-4850-ae16-2fbbf4af32f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:21.486043 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fef854-0be0-4850-ae16-2fbbf4af32f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:56:21.486347 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fef854-0be0-4850-ae16-2fbbf4af32f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:21.487904 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fef854-0be0-4850-ae16-2fbbf4af32f3 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:21.496456 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 588/1181] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:56:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:56:21.502228 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb469dc-b19c-4660-ab99-e84266bb2a68 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:21.535011 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:21.612589 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb469dc-b19c-4660-ab99-e84266bb2a68 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:21.645817 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758197a-4350-4c7a-b084-9a45854f76d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:21.646456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758197a-4350-4c7a-b084-9a45854f76d5 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:21.646734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758197a-4350-4c7a-b084-9a45854f76d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:21.647040 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758197a-4350-4c7a-b084-9a45854f76d5 None admin] PATH_INFO: `/v3/users/80b88b6587a24966b9e180ad487b6d9e` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:21.660433 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4758197a-4350-4c7a-b084-9a45854f76d5 None admin] RBAC: Authorizing `identity:get_user(user_id=80b88b6587a24966b9e180ad487b6d9e)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:21.661894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4758197a-4350-4c7a-b084-9a45854f76d5 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:21.663582 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 594/1182] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:56:21 2022] GET /identity/v3/users/80b88b6587a24966b9e180ad487b6d9e => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:56:21.669943 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fef854-0be0-4850-ae16-2fbbf4af32f3 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:21.731015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:21.769629 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fef854-0be0-4850-ae16-2fbbf4af32f3 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:21.801548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7624c-25ca-428e-9fa2-768445a704b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:21.802150 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7624c-25ca-428e-9fa2-768445a704b1 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:21.802431 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7624c-25ca-428e-9fa2-768445a704b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:21.802705 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7624c-25ca-428e-9fa2-768445a704b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:21.872776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7624c-25ca-428e-9fa2-768445a704b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:21.874150 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7624c-25ca-428e-9fa2-768445a704b1 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:21.933289 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:21.970294 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 589/1183] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:56:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 04:56:21.976230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758197a-4350-4c7a-b084-9a45854f76d5 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:22.057686 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:22.075093 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758197a-4350-4c7a-b084-9a45854f76d5 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:22.108669 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194c137-1682-452f-bf52-e1ba83a682f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:22.109328 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194c137-1682-452f-bf52-e1ba83a682f1 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:22.109613 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194c137-1682-452f-bf52-e1ba83a682f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:22.109895 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194c137-1682-452f-bf52-e1ba83a682f1 None admin] PATH_INFO: `/v3/projects/379c974703a44ba8a0faa86c7bdfe7eb` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:22.111072 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194c137-1682-452f-bf52-e1ba83a682f1 None admin] RBAC: Authorizing `identity:get_project(project_id=379c974703a44ba8a0faa86c7bdfe7eb)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:22.112508 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194c137-1682-452f-bf52-e1ba83a682f1 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:22.114143 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 595/1184] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:56:21 2022] GET /identity/v3/projects/379c974703a44ba8a0faa86c7bdfe7eb => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:56:22.120844 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7624c-25ca-428e-9fa2-768445a704b1 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:22.218190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7624c-25ca-428e-9fa2-768445a704b1 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:22.250653 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af26c7-4cbf-41cf-b453-a9714e4a7d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:22.251282 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af26c7-4cbf-41cf-b453-a9714e4a7d2c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:22.251561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af26c7-4cbf-41cf-b453-a9714e4a7d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:22.251833 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af26c7-4cbf-41cf-b453-a9714e4a7d2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:22.252614 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af26c7-4cbf-41cf-b453-a9714e4a7d2c None admin] RBAC: Adding query filter params (role.id=e39d4ecea4c64d5794e1079c9be3ad36, scope.project.id=379c974703a44ba8a0faa86c7bdfe7eb, user.id=80b88b6587a24966b9e180ad487b6d9e) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:56:22.252923 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af26c7-4cbf-41cf-b453-a9714e4a7d2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:22.254341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af26c7-4cbf-41cf-b453-a9714e4a7d2c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:22.260192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 590/1185] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:56:22 2022] GET /identity/v3/role_assignments?user.id=80b88b6587a24966b9e180ad487b6d9e&scope.project.id=379c974703a44ba8a0faa86c7bdfe7eb&role.id=e39d4ecea4c64d5794e1079c9be3ad36 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:56:24.232596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd8d00-9ef5-43b5-9a2a-515833744f8a None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:24.233120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd8d00-9ef5-43b5-9a2a-515833744f8a None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:24.233211 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd8d00-9ef5-43b5-9a2a-515833744f8a None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:24.234491 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 596/1186] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:56:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:56:24.239746 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e629cd-6df8-4da7-b29e-9901120975dc None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:24.240177 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e629cd-6df8-4da7-b29e-9901120975dc None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:24.240305 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e629cd-6df8-4da7-b29e-9901120975dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:24.258653 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-a7e629cd-6df8-4da7-b29e-9901120975dc None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:56:24.259864 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a7e629cd-6df8-4da7-b29e-9901120975dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:56:24.279562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:24.325077 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 591/1187] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:56:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:56:24.347274 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd8d00-9ef5-43b5-9a2a-515833744f8a None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:24.447329 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd8d00-9ef5-43b5-9a2a-515833744f8a None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:24.480124 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cbbf95-17cf-4326-bbd5-55c25034fb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:24.480882 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbbf95-17cf-4326-bbd5-55c25034fb66 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:24.481035 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbbf95-17cf-4326-bbd5-55c25034fb66 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:24.481306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbbf95-17cf-4326-bbd5-55c25034fb66 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:24.491255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cbbf95-17cf-4326-bbd5-55c25034fb66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:56:24.496549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-60cbbf95-17cf-4326-bbd5-55c25034fb66 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:56:24.496549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:56:24.496549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:56:24.496549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:56:24.496549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:56:24.496549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:56:24.496549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:56:24.496549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:56:24.496549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:56:24.496549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:56:24.496549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:56:24.496549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:56:24.496549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:56:24.496549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:56:24.496549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:56:24.496549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:56:24.497649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:56:24.497649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:56:24.497649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:56:24.497649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:56:24.497649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:56:24.497649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:56:24.497649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:56:24.497649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:56:24.497649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:56:24.497649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:56:24.497649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:56:24.497649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:56:24.497649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:56:24.497649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:56:24.497649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:56:24.497649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:56:24.498554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:56:24.498554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:56:24.498554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:56:24.498554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:56:24.498554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:56:24.498554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:56:24.498554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:56:24.498554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:56:24.498554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:56:24.498554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:56:24.498554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:56:24.498554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:56:24.498554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 597/1188] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:56:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:56:24.503712 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e629cd-6df8-4da7-b29e-9901120975dc None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:24.600790 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e629cd-6df8-4da7-b29e-9901120975dc None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:24.633738 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63fe28-5b7f-4391-b7c8-9a1ca1b51256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:24.634349 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63fe28-5b7f-4391-b7c8-9a1ca1b51256 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:24.634639 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63fe28-5b7f-4391-b7c8-9a1ca1b51256 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:24.634950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63fe28-5b7f-4391-b7c8-9a1ca1b51256 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:24.635663 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c63fe28-5b7f-4391-b7c8-9a1ca1b51256 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:56:24.635971 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c63fe28-5b7f-4391-b7c8-9a1ca1b51256 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:24.637337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c63fe28-5b7f-4391-b7c8-9a1ca1b51256 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:24.645723 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 592/1189] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:56:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:56:24.651965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cbbf95-17cf-4326-bbd5-55c25034fb66 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:24.665452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:24.750457 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cbbf95-17cf-4326-bbd5-55c25034fb66 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:24.784001 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d1782-d55f-4aae-aa58-bbdf5b423181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:24.784636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d1782-d55f-4aae-aa58-bbdf5b423181 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:24.784920 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d1782-d55f-4aae-aa58-bbdf5b423181 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:24.785197 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d1782-d55f-4aae-aa58-bbdf5b423181 None admin] PATH_INFO: `/v3/users/80b88b6587a24966b9e180ad487b6d9e` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:24.786354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d1782-d55f-4aae-aa58-bbdf5b423181 None admin] RBAC: Authorizing `identity:get_user(user_id=80b88b6587a24966b9e180ad487b6d9e)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:24.787824 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d1782-d55f-4aae-aa58-bbdf5b423181 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:24.789412 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 598/1190] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:56:24 2022] GET /identity/v3/users/80b88b6587a24966b9e180ad487b6d9e => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:56:24.795871 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63fe28-5b7f-4391-b7c8-9a1ca1b51256 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:24.815753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:24.894829 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63fe28-5b7f-4391-b7c8-9a1ca1b51256 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:24.927820 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbde6a8-b3af-45b6-9912-289a793ab91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:24.928452 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbde6a8-b3af-45b6-9912-289a793ab91f None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:24.928736 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbde6a8-b3af-45b6-9912-289a793ab91f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:24.929012 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbde6a8-b3af-45b6-9912-289a793ab91f None admin] PATH_INFO: `/v3/projects/379c974703a44ba8a0faa86c7bdfe7eb` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:24.930154 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbde6a8-b3af-45b6-9912-289a793ab91f None admin] RBAC: Authorizing `identity:get_project(project_id=379c974703a44ba8a0faa86c7bdfe7eb)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:24.931639 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbde6a8-b3af-45b6-9912-289a793ab91f None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:24.933299 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 593/1191] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:56:24 2022] GET /identity/v3/projects/379c974703a44ba8a0faa86c7bdfe7eb => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:56:24.939471 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d1782-d55f-4aae-aa58-bbdf5b423181 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:25.025118 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:25.045820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d1782-d55f-4aae-aa58-bbdf5b423181 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:25.079023 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e32d4e2-5ca2-42b6-838b-dcfeb22e028d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:25.079683 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32d4e2-5ca2-42b6-838b-dcfeb22e028d None admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:25.079971 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32d4e2-5ca2-42b6-838b-dcfeb22e028d None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:25.080247 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32d4e2-5ca2-42b6-838b-dcfeb22e028d None admin] PATH_INFO: `/v3/projects/379c974703a44ba8a0faa86c7bdfe7eb/users/80b88b6587a24966b9e180ad487b6d9e/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:25.082152 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e32d4e2-5ca2-42b6-838b-dcfeb22e028d None admin] RBAC: Authorizing `identity:create_grant(project_id=379c974703a44ba8a0faa86c7bdfe7eb, user_id=80b88b6587a24966b9e180ad487b6d9e, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:25.083669 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e32d4e2-5ca2-42b6-838b-dcfeb22e028d None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:25.157078 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 599/1192] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 3 04:56:24 2022] PUT /identity/v3/projects/379c974703a44ba8a0faa86c7bdfe7eb/users/80b88b6587a24966b9e180ad487b6d9e/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 04:56:27.142662 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfeeaea-7beb-4a02-9a85-40dce494b669 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:27.143296 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfeeaea-7beb-4a02-9a85-40dce494b669 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:27.143296 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfeeaea-7beb-4a02-9a85-40dce494b669 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:27.144871 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 594/1193] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:56:27.150675 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23997e56-c862-44e1-8d1b-b7e2cb55a71d None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:27.151145 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23997e56-c862-44e1-8d1b-b7e2cb55a71d None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:27.151277 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23997e56-c862-44e1-8d1b-b7e2cb55a71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:27.169719 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-23997e56-c862-44e1-8d1b-b7e2cb55a71d None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:56:27.170953 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-23997e56-c862-44e1-8d1b-b7e2cb55a71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:56:27.235984 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 600/1194] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:56:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:56:27.259391 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfeeaea-7beb-4a02-9a85-40dce494b669 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:27.343460 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:27.358502 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfeeaea-7beb-4a02-9a85-40dce494b669 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:27.391814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca35237-0b1a-435e-abfd-0f204753ff01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:27.392436 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca35237-0b1a-435e-abfd-0f204753ff01 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:27.392763 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca35237-0b1a-435e-abfd-0f204753ff01 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:27.392985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca35237-0b1a-435e-abfd-0f204753ff01 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:27.403911 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca35237-0b1a-435e-abfd-0f204753ff01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:56:27.408881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-aca35237-0b1a-435e-abfd-0f204753ff01 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:56:27.408881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:56:27.408881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:56:27.408881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:56:27.408881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:56:27.408881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:56:27.408881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:56:27.408881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:56:27.408881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:56:27.408881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:56:27.408881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:56:27.408881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:56:27.408881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:56:27.408881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:56:27.408881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:56:27.408881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:56:27.410098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:56:27.410098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:56:27.410098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:56:27.410098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:56:27.410098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:56:27.410098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:56:27.410098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:56:27.410098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:56:27.410098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:56:27.410098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:56:27.410098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:56:27.410098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:56:27.410098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:56:27.410098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:56:27.410098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:56:27.410098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:56:27.411197 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:56:27.411197 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:56:27.411197 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:56:27.411197 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:56:27.411197 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:56:27.411197 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:56:27.411197 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:56:27.411197 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:56:27.411197 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:56:27.411197 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:56:27.411197 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:56:27.411197 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:56:27.411197 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 595/1195] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:56:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:56:27.416138 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23997e56-c862-44e1-8d1b-b7e2cb55a71d None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:27.481987 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:27.514053 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23997e56-c862-44e1-8d1b-b7e2cb55a71d None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:27.547790 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55aa641-9153-401a-ace8-ab2c2437209a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:27.548397 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55aa641-9153-401a-ace8-ab2c2437209a None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:27.548676 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55aa641-9153-401a-ace8-ab2c2437209a None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:27.548945 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55aa641-9153-401a-ace8-ab2c2437209a None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:27.549644 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55aa641-9153-401a-ace8-ab2c2437209a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:56:27.549946 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55aa641-9153-401a-ace8-ab2c2437209a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:27.551344 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55aa641-9153-401a-ace8-ab2c2437209a None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:27.559674 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 601/1196] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:56:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:56:27.565458 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca35237-0b1a-435e-abfd-0f204753ff01 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:27.664024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca35237-0b1a-435e-abfd-0f204753ff01 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:27.697285 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d9c71-443a-4a54-abd1-3dee4d7c8bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:27.697911 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d9c71-443a-4a54-abd1-3dee4d7c8bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:27.698192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d9c71-443a-4a54-abd1-3dee4d7c8bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:27.698467 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d9c71-443a-4a54-abd1-3dee4d7c8bf8 None admin] PATH_INFO: `/v3/users/80b88b6587a24966b9e180ad487b6d9e` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:27.699643 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6d9c71-443a-4a54-abd1-3dee4d7c8bf8 None admin] RBAC: Authorizing `identity:get_user(user_id=80b88b6587a24966b9e180ad487b6d9e)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:27.701066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6d9c71-443a-4a54-abd1-3dee4d7c8bf8 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:27.702654 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 596/1197] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Aug 3 04:56:27 2022] GET /identity/v3/users/80b88b6587a24966b9e180ad487b6d9e => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:56:27.708527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55aa641-9153-401a-ace8-ab2c2437209a None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:27.805766 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55aa641-9153-401a-ace8-ab2c2437209a None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:27.833067 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:27.840513 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d19cb-e615-40db-95d0-db27a35dc2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:27.843651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d19cb-e615-40db-95d0-db27a35dc2f8 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:27.843651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d19cb-e615-40db-95d0-db27a35dc2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:27.843651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d19cb-e615-40db-95d0-db27a35dc2f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:27.907383 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151d19cb-e615-40db-95d0-db27a35dc2f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:27.908832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151d19cb-e615-40db-95d0-db27a35dc2f8 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:28.009225 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 602/1198] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:56:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 04:56:28.015343 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d9c71-443a-4a54-abd1-3dee4d7c8bf8 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:28.031314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:28.113283 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d9c71-443a-4a54-abd1-3dee4d7c8bf8 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:28.146196 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911379bb-807e-49ca-861d-87dbc10262a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:28.146819 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911379bb-807e-49ca-861d-87dbc10262a9 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:28.147128 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911379bb-807e-49ca-861d-87dbc10262a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:28.147401 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911379bb-807e-49ca-861d-87dbc10262a9 None admin] PATH_INFO: `/v3/projects/379c974703a44ba8a0faa86c7bdfe7eb` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:28.148532 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911379bb-807e-49ca-861d-87dbc10262a9 None admin] RBAC: Authorizing `identity:get_project(project_id=379c974703a44ba8a0faa86c7bdfe7eb)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:28.149982 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911379bb-807e-49ca-861d-87dbc10262a9 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:28.151711 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 597/1199] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Aug 3 04:56:28 2022] GET /identity/v3/projects/379c974703a44ba8a0faa86c7bdfe7eb => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:56:28.157775 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d19cb-e615-40db-95d0-db27a35dc2f8 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:56:28.187690 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:28.254859 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d19cb-e615-40db-95d0-db27a35dc2f8 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:56:28.288036 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993e7af-4e81-478f-94c2-7a34840e0687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:56:28.288584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993e7af-4e81-478f-94c2-7a34840e0687 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:56:28.288864 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993e7af-4e81-478f-94c2-7a34840e0687 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:56:28.289136 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993e7af-4e81-478f-94c2-7a34840e0687 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:56:28.290478 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993e7af-4e81-478f-94c2-7a34840e0687 None admin] RBAC: Adding query filter params (role.id=e39d4ecea4c64d5794e1079c9be3ad36, scope.project.id=379c974703a44ba8a0faa86c7bdfe7eb, user.id=80b88b6587a24966b9e180ad487b6d9e) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:56:28.290782 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993e7af-4e81-478f-94c2-7a34840e0687 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:56:28.292243 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993e7af-4e81-478f-94c2-7a34840e0687 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:56:28.297839 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 603/1200] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:56:28 2022] GET /identity/v3/role_assignments?user.id=80b88b6587a24966b9e180ad487b6d9e&scope.project.id=379c974703a44ba8a0faa86c7bdfe7eb&role.id=e39d4ecea4c64d5794e1079c9be3ad36 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:56:38.141897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=59 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:56:38.295303 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=53 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:39.954348 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e0f36-f7b1-4928-88ad-3f6d7212d005 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:39.955124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e0f36-f7b1-4928-88ad-3f6d7212d005 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:39.955124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e0f36-f7b1-4928-88ad-3f6d7212d005 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:39.956223 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 598/1201] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:57:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:57:39.961433 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f23d52-2a5d-42af-8ae4-76f3557aebc0 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:39.961862 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f23d52-2a5d-42af-8ae4-76f3557aebc0 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:39.961993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f23d52-2a5d-42af-8ae4-76f3557aebc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:39.980631 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-13f23d52-2a5d-42af-8ae4-76f3557aebc0 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:57:39.984282 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-13f23d52-2a5d-42af-8ae4-76f3557aebc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:57:40.047972 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 604/1202] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:57:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:57:40.072512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e0f36-f7b1-4928-88ad-3f6d7212d005 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:40.175188 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e0f36-f7b1-4928-88ad-3f6d7212d005 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:40.209154 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8f430-fcbd-4c2e-8915-bd70d735d01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:40.209733 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8f430-fcbd-4c2e-8915-bd70d735d01a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:40.210048 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8f430-fcbd-4c2e-8915-bd70d735d01a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:40.210325 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8f430-fcbd-4c2e-8915-bd70d735d01a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:40.220822 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf8f430-fcbd-4c2e-8915-bd70d735d01a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:40.222328 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf8f430-fcbd-4c2e-8915-bd70d735d01a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:40.237944 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-bbf8f430-fcbd-4c2e-8915-bd70d735d01a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:57:40.237944 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:57:40.237944 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:57:40.237944 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:57:40.237944 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:57:40.237944 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:57:40.237944 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:57:40.237944 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:57:40.237944 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:57:40.237944 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:57:40.237944 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:57:40.237944 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:57:40.237944 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:57:40.237944 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:57:40.237944 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:57:40.242973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:57:40.242973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:57:40.242973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:57:40.242973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:57:40.242973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:57:40.242973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:57:40.242973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:57:40.242973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:57:40.242973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:57:40.242973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:57:40.242973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:57:40.242973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:57:40.242973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:57:40.242973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:57:40.242973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:57:40.242973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:57:40.242973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:57:40.244248 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:57:40.244248 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:57:40.244248 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:57:40.244248 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:57:40.244248 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 599/1203] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:57:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:57:40.245333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f23d52-2a5d-42af-8ae4-76f3557aebc0 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:40.346657 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f23d52-2a5d-42af-8ae4-76f3557aebc0 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:40.374826 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:40.380474 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973dc7e-2537-4e2e-bd7a-380c89534df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:40.381085 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973dc7e-2537-4e2e-bd7a-380c89534df2 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:40.381367 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973dc7e-2537-4e2e-bd7a-380c89534df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:40.381638 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973dc7e-2537-4e2e-bd7a-380c89534df2 None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:40.382316 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b973dc7e-2537-4e2e-bd7a-380c89534df2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:57:40.382616 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b973dc7e-2537-4e2e-bd7a-380c89534df2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:40.384013 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b973dc7e-2537-4e2e-bd7a-380c89534df2 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:40.395122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 605/1204] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:57:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:57:40.401320 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8f430-fcbd-4c2e-8915-bd70d735d01a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:40.453685 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:40.502751 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8f430-fcbd-4c2e-8915-bd70d735d01a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:40.536800 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cedbc98-f932-49ab-94e6-df225be9618c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:40.537412 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedbc98-f932-49ab-94e6-df225be9618c None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:40.537687 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedbc98-f932-49ab-94e6-df225be9618c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:40.537958 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedbc98-f932-49ab-94e6-df225be9618c None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:40.538641 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cedbc98-f932-49ab-94e6-df225be9618c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:40.540100 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cedbc98-f932-49ab-94e6-df225be9618c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:40.708932 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 600/1205] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Aug 3 04:57:40 2022] POST /identity/v3/users => generated 251 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:57:42.769327 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693f80c-8354-4923-89e2-d50d15ef4d28 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:42.769882 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693f80c-8354-4923-89e2-d50d15ef4d28 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:42.769882 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693f80c-8354-4923-89e2-d50d15ef4d28 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:42.771287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 606/1206] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:57:42.776396 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d463b9-a14a-453d-aa8e-af570a160a1a None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:42.776795 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d463b9-a14a-453d-aa8e-af570a160a1a None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:42.777089 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d463b9-a14a-453d-aa8e-af570a160a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:42.796013 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-a3d463b9-a14a-453d-aa8e-af570a160a1a None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:57:42.797201 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a3d463b9-a14a-453d-aa8e-af570a160a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:57:42.863131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 601/1207] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:57:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:57:42.886527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693f80c-8354-4923-89e2-d50d15ef4d28 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:43.008900 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693f80c-8354-4923-89e2-d50d15ef4d28 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:43.042541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35004e0-6da2-4b52-ad55-b2f89ccffee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:43.043211 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35004e0-6da2-4b52-ad55-b2f89ccffee4 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:43.043499 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35004e0-6da2-4b52-ad55-b2f89ccffee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:43.043777 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35004e0-6da2-4b52-ad55-b2f89ccffee4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:43.055124 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35004e0-6da2-4b52-ad55-b2f89ccffee4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:57:43.060695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-f35004e0-6da2-4b52-ad55-b2f89ccffee4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:57:43.060695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:57:43.060695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:57:43.060695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:57:43.060695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:57:43.060695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:57:43.060695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:57:43.060695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:57:43.060695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:57:43.060695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:57:43.060695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:57:43.060695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:57:43.060695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:57:43.060695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:57:43.060695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:57:43.060695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:57:43.061819 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:57:43.061819 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:57:43.061819 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:57:43.061819 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:57:43.061819 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:57:43.061819 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:57:43.061819 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:57:43.061819 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:57:43.061819 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:57:43.061819 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:57:43.061819 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:57:43.061819 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:57:43.061819 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:57:43.061819 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:57:43.061819 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:57:43.061819 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:57:43.062731 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:57:43.062731 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:57:43.062731 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:57:43.062731 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:57:43.062731 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:57:43.062731 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:57:43.062731 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:57:43.062731 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:57:43.062731 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:57:43.062731 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:57:43.062731 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:57:43.062731 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:57:43.062731 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 607/1208] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 3 04:57:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:57:43.068088 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d463b9-a14a-453d-aa8e-af570a160a1a None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:43.106819 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:43.164012 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d463b9-a14a-453d-aa8e-af570a160a1a None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:43.196955 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a828da2-3449-4668-b654-0d709b919412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:43.197602 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a828da2-3449-4668-b654-0d709b919412 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:43.197876 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a828da2-3449-4668-b654-0d709b919412 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:43.198153 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a828da2-3449-4668-b654-0d709b919412 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:43.198891 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a828da2-3449-4668-b654-0d709b919412 None admin] RBAC: Adding query filter params (name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:57:43.199204 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a828da2-3449-4668-b654-0d709b919412 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:43.200572 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a828da2-3449-4668-b654-0d709b919412 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:43.208380 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 602/1209] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 3 04:57:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:57:43.214834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35004e0-6da2-4b52-ad55-b2f89ccffee4 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:43.268340 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:43.315988 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35004e0-6da2-4b52-ad55-b2f89ccffee4 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:43.348077 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab803d57-8c20-4c4e-8d65-766a0522942e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:43.348705 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab803d57-8c20-4c4e-8d65-766a0522942e None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:43.349066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab803d57-8c20-4c4e-8d65-766a0522942e None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:43.349253 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab803d57-8c20-4c4e-8d65-766a0522942e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:43.359322 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab803d57-8c20-4c4e-8d65-766a0522942e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:43.360762 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab803d57-8c20-4c4e-8d65-766a0522942e None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:43.374699 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-ab803d57-8c20-4c4e-8d65-766a0522942e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:57:43.374699 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:57:43.374699 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:57:43.374699 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:57:43.374699 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:57:43.374699 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:57:43.374699 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:57:43.374699 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:57:43.374699 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:57:43.374699 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:57:43.374699 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:57:43.374699 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:57:43.374699 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:57:43.374699 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:57:43.374699 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:57:43.377869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:57:43.377869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:57:43.377869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:57:43.377869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:57:43.377869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:57:43.377869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:57:43.377869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:57:43.377869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:57:43.377869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:57:43.377869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:57:43.377869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:57:43.377869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:57:43.377869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:57:43.377869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:57:43.377869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:57:43.377869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:57:43.377869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:57:43.378759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:57:43.378759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:57:43.378759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:57:43.378759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:57:43.378759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 608/1210] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:57:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:57:43.381846 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a828da2-3449-4668-b654-0d709b919412 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:43.479690 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:43.482561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a828da2-3449-4668-b654-0d709b919412 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:43.516360 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44b597-fc19-409b-8b71-ff0a4506b644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:43.516967 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44b597-fc19-409b-8b71-ff0a4506b644 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:43.517249 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44b597-fc19-409b-8b71-ff0a4506b644 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:43.517524 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44b597-fc19-409b-8b71-ff0a4506b644 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:43.518204 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b44b597-fc19-409b-8b71-ff0a4506b644 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:57:43.518558 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b44b597-fc19-409b-8b71-ff0a4506b644 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:43.519912 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b44b597-fc19-409b-8b71-ff0a4506b644 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:43.531095 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 603/1211] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:57:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:57:43.536838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab803d57-8c20-4c4e-8d65-766a0522942e None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:43.638089 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab803d57-8c20-4c4e-8d65-766a0522942e None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:43.640231 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:43.673999 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e31e52-09d9-453c-b6fc-adce263ca1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:43.678583 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e31e52-09d9-453c-b6fc-adce263ca1fd None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:43.678583 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e31e52-09d9-453c-b6fc-adce263ca1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:43.678583 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e31e52-09d9-453c-b6fc-adce263ca1fd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:43.683697 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e31e52-09d9-453c-b6fc-adce263ca1fd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:43.685142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e31e52-09d9-453c-b6fc-adce263ca1fd None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:43.696068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-75e31e52-09d9-453c-b6fc-adce263ca1fd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 03 04:57:43.696068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:57:43.696068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:57:43.696068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:57:43.696068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:57:43.696068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:57:43.696068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:57:43.696068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:57:43.696068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:57:43.696068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:57:43.696068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:57:43.696068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:57:43.696068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:57:43.696068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:57:43.696068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:57:43.696068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:57:43.702142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:57:43.702142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:57:43.702142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:57:43.702142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:57:43.702142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:57:43.702142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:57:43.702142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:57:43.702142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:57:43.702142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:57:43.702142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:57:43.702142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:57:43.702142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:57:43.702142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:57:43.702142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:57:43.702142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:57:43.702142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:57:43.702142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:57:43.702142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:57:43.703363 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:57:43.703363 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:57:43.703363 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:57:43.703363 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:57:43.703363 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:57:43.703363 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:57:43.703363 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:57:43.703363 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:57:43.703363 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 03 04:57:43.703363 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:57:43.703363 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 609/1212] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:57:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:57:43.704122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44b597-fc19-409b-8b71-ff0a4506b644 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:43.814201 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44b597-fc19-409b-8b71-ff0a4506b644 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:43.850101 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66160a3-9c34-423c-ae04-5be740f941a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:43.850762 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66160a3-9c34-423c-ae04-5be740f941a8 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:43.851071 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66160a3-9c34-423c-ae04-5be740f941a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:43.851347 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66160a3-9c34-423c-ae04-5be740f941a8 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:43.852133 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66160a3-9c34-423c-ae04-5be740f941a8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:57:43.852437 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66160a3-9c34-423c-ae04-5be740f941a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:43.853902 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66160a3-9c34-423c-ae04-5be740f941a8 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:43.872845 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 604/1213] 10.222.0.40 () {56 vars in 1143 bytes} [Wed Aug 3 04:57:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:57:43.878719 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e31e52-09d9-453c-b6fc-adce263ca1fd None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:43.980530 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e31e52-09d9-453c-b6fc-adce263ca1fd None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:44.014892 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a817b32-ece3-41a9-b70c-4f37aae956ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:44.015514 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a817b32-ece3-41a9-b70c-4f37aae956ec None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:44.016049 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a817b32-ece3-41a9-b70c-4f37aae956ec None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:44.016325 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a817b32-ece3-41a9-b70c-4f37aae956ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:44.045718 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:44.083489 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a817b32-ece3-41a9-b70c-4f37aae956ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:44.084932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a817b32-ece3-41a9-b70c-4f37aae956ec None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:44.183011 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 610/1214] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:57:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 04:57:44.188973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66160a3-9c34-423c-ae04-5be740f941a8 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:44.208386 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:44.289056 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66160a3-9c34-423c-ae04-5be740f941a8 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:44.323147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739caf9d-96d5-4bcc-b812-6e7173fc4a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:44.323779 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739caf9d-96d5-4bcc-b812-6e7173fc4a2a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:44.324063 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739caf9d-96d5-4bcc-b812-6e7173fc4a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:44.324329 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739caf9d-96d5-4bcc-b812-6e7173fc4a2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:44.334345 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739caf9d-96d5-4bcc-b812-6e7173fc4a2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:44.335769 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739caf9d-96d5-4bcc-b812-6e7173fc4a2a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:44.350144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-739caf9d-96d5-4bcc-b812-6e7173fc4a2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:57:44.350144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:57:44.350144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:57:44.350144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:57:44.350144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:57:44.350144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:57:44.350144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:57:44.350144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:57:44.350144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:57:44.350144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:57:44.350144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:57:44.350144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:57:44.350144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:57:44.350144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:57:44.350144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:57:44.351271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:57:44.351271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:57:44.351271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:57:44.351271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:57:44.351271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:57:44.351271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:57:44.351271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:57:44.351271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:57:44.351271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:57:44.351271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:57:44.351271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:57:44.351271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:57:44.351271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:57:44.351271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:57:44.351271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:57:44.351271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:57:44.351271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:57:44.352356 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:57:44.352356 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:57:44.352356 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:57:44.352356 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:57:44.359043 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 605/1215] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:57:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:57:44.361614 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a817b32-ece3-41a9-b70c-4f37aae956ec None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:44.419241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:44.457918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a817b32-ece3-41a9-b70c-4f37aae956ec None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:44.490950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab5bca-0ee2-415b-b300-d0a73913868e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:44.491561 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab5bca-0ee2-415b-b300-d0a73913868e None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:44.491845 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab5bca-0ee2-415b-b300-d0a73913868e None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:44.492122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab5bca-0ee2-415b-b300-d0a73913868e None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:44.492813 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab5bca-0ee2-415b-b300-d0a73913868e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:57:44.493117 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab5bca-0ee2-415b-b300-d0a73913868e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:44.495141 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab5bca-0ee2-415b-b300-d0a73913868e None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:44.506058 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 611/1216] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:57:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:57:44.511887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739caf9d-96d5-4bcc-b812-6e7173fc4a2a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:44.582610 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:44.612780 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739caf9d-96d5-4bcc-b812-6e7173fc4a2a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:44.647110 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541107e-5616-4340-9a6d-80750739d2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:44.657581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541107e-5616-4340-9a6d-80750739d2a7 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:44.657581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541107e-5616-4340-9a6d-80750739d2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:44.657581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541107e-5616-4340-9a6d-80750739d2a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:44.657581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d541107e-5616-4340-9a6d-80750739d2a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:44.657581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d541107e-5616-4340-9a6d-80750739d2a7 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:44.669079 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-d541107e-5616-4340-9a6d-80750739d2a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:57:44.669079 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:57:44.669079 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:57:44.669079 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:57:44.669079 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:57:44.669079 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:57:44.669079 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:57:44.669079 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:57:44.669079 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:57:44.669079 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:57:44.669079 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:57:44.669079 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:57:44.669079 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:57:44.669079 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:57:44.669079 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:57:44.670173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:57:44.670173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:57:44.670173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:57:44.670173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:57:44.670173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:57:44.670173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:57:44.670173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:57:44.670173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:57:44.670173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:57:44.670173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:57:44.670173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:57:44.670173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:57:44.670173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:57:44.670173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:57:44.670173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:57:44.670173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:57:44.670173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:57:44.671551 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:57:44.671551 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:57:44.671551 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:57:44.671551 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:57:44.671551 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:57:44.671551 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:57:44.671551 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:57:44.671551 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:57:44.671551 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 606/1217] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:57:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 04:57:44.676047 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab5bca-0ee2-415b-b300-d0a73913868e None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:44.774334 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab5bca-0ee2-415b-b300-d0a73913868e None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:44.797425 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:44.808603 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e7bcc-df16-4792-aa4d-70fc98f9e490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:44.809211 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e7bcc-df16-4792-aa4d-70fc98f9e490 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:44.809495 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e7bcc-df16-4792-aa4d-70fc98f9e490 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:44.809793 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e7bcc-df16-4792-aa4d-70fc98f9e490 None admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:44.810560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e7bcc-df16-4792-aa4d-70fc98f9e490 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:57:44.810887 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e7bcc-df16-4792-aa4d-70fc98f9e490 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:44.812329 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e7bcc-df16-4792-aa4d-70fc98f9e490 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:44.824038 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 612/1218] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:57:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:57:44.830306 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541107e-5616-4340-9a6d-80750739d2a7 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:44.931389 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541107e-5616-4340-9a6d-80750739d2a7 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:44.958177 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:44.964507 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4314d54d-66ce-453e-a1f7-06b38b0ff7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:44.965115 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314d54d-66ce-453e-a1f7-06b38b0ff7d9 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:44.965396 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314d54d-66ce-453e-a1f7-06b38b0ff7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:44.965675 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314d54d-66ce-453e-a1f7-06b38b0ff7d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:44.966446 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4314d54d-66ce-453e-a1f7-06b38b0ff7d9 None admin] RBAC: Adding query filter params (role.id=cae571342db14f5fac7849efe7f8636e, scope.project.id=cf3763a37383456184a9ca9351b4ce03, user.id=59846d88f4314c2382aa5cb8bae5053b) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:57:44.966753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4314d54d-66ce-453e-a1f7-06b38b0ff7d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:44.968209 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4314d54d-66ce-453e-a1f7-06b38b0ff7d9 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:44.973529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 607/1219] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:57:44 2022] GET /identity/v3/role_assignments?user.id=59846d88f4314c2382aa5cb8bae5053b&scope.project.id=cf3763a37383456184a9ca9351b4ce03&role.id=cae571342db14f5fac7849efe7f8636e => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:57:46.979181 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ac456-8349-4b9d-bf23-d820b7fb4e60 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:46.979710 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ac456-8349-4b9d-bf23-d820b7fb4e60 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:46.979710 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ac456-8349-4b9d-bf23-d820b7fb4e60 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:46.980987 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 613/1220] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:57:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:57:46.986286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65283d90-0af9-4151-b410-c1ddde1ae1a9 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:46.986677 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65283d90-0af9-4151-b410-c1ddde1ae1a9 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:46.986882 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65283d90-0af9-4151-b410-c1ddde1ae1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:47.004722 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-65283d90-0af9-4151-b410-c1ddde1ae1a9 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:57:47.005909 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-65283d90-0af9-4151-b410-c1ddde1ae1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:57:47.072663 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 608/1221] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:57:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:57:47.095827 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17ac456-8349-4b9d-bf23-d820b7fb4e60 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:47.198096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17ac456-8349-4b9d-bf23-d820b7fb4e60 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:47.232372 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e19e8-a4ea-4be1-b284-58d1c31465e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:47.232936 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e19e8-a4ea-4be1-b284-58d1c31465e0 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:47.233214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e19e8-a4ea-4be1-b284-58d1c31465e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:47.233486 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e19e8-a4ea-4be1-b284-58d1c31465e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:47.243257 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e19e8-a4ea-4be1-b284-58d1c31465e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:57:47.248200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-758e19e8-a4ea-4be1-b284-58d1c31465e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:57:47.248200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:57:47.248200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:57:47.248200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:57:47.248200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:57:47.248200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:57:47.248200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:57:47.248200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:57:47.248200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:57:47.248200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:57:47.248200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:57:47.248200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:57:47.248200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:57:47.248200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:57:47.248200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:57:47.248200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:57:47.249322 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:57:47.249322 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:57:47.249322 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:57:47.249322 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:57:47.249322 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:57:47.249322 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:57:47.249322 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:57:47.249322 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:57:47.249322 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:57:47.249322 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:57:47.249322 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:57:47.249322 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:57:47.249322 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:57:47.249322 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:57:47.249322 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:57:47.249322 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:57:47.250193 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:57:47.250193 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:57:47.250193 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:57:47.250193 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:57:47.250193 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:57:47.250193 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:57:47.250193 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:57:47.250193 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:57:47.250193 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:57:47.250193 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:57:47.250193 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:57:47.250193 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:57:47.254384 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 614/1222] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 3 04:57:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:57:47.261277 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65283d90-0af9-4151-b410-c1ddde1ae1a9 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:47.356159 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65283d90-0af9-4151-b410-c1ddde1ae1a9 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:47.380173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:47.391284 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be1b64f-434d-483f-a26e-2dead114e234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:47.391894 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1b64f-434d-483f-a26e-2dead114e234 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:47.392175 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1b64f-434d-483f-a26e-2dead114e234 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:47.392449 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1b64f-434d-483f-a26e-2dead114e234 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:47.393153 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be1b64f-434d-483f-a26e-2dead114e234 None admin] RBAC: Adding query filter params (name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:57:47.393455 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be1b64f-434d-483f-a26e-2dead114e234 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:47.394952 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be1b64f-434d-483f-a26e-2dead114e234 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:47.403331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 609/1223] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 3 04:57:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:57:47.409121 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e19e8-a4ea-4be1-b284-58d1c31465e0 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:47.453374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:47.508372 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e19e8-a4ea-4be1-b284-58d1c31465e0 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:47.543326 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac91c79a-e0bd-4b4a-9847-fdf946433972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:47.543965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91c79a-e0bd-4b4a-9847-fdf946433972 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:47.544248 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91c79a-e0bd-4b4a-9847-fdf946433972 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:47.544528 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91c79a-e0bd-4b4a-9847-fdf946433972 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:47.554604 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac91c79a-e0bd-4b4a-9847-fdf946433972 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:47.556211 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac91c79a-e0bd-4b4a-9847-fdf946433972 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:47.570817 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-ac91c79a-e0bd-4b4a-9847-fdf946433972 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:57:47.570817 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:57:47.570817 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:57:47.570817 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:57:47.570817 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:57:47.570817 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:57:47.570817 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:57:47.570817 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:57:47.570817 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:57:47.570817 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:57:47.570817 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:57:47.570817 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:57:47.570817 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:57:47.570817 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:57:47.570817 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:57:47.571918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:57:47.571918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:57:47.571918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:57:47.571918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:57:47.571918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:57:47.571918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:57:47.571918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:57:47.571918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:57:47.571918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:57:47.571918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:57:47.571918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:57:47.571918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:57:47.571918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:57:47.571918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:57:47.571918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:57:47.571918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:57:47.571918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:57:47.573451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:57:47.573451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:57:47.573451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:57:47.573451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:57:47.573451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 615/1224] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:57:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:57:47.578454 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be1b64f-434d-483f-a26e-2dead114e234 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:47.680098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be1b64f-434d-483f-a26e-2dead114e234 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:47.721128 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b5199-b9af-4255-9f16-6b8e9e80055f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:47.721742 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b5199-b9af-4255-9f16-6b8e9e80055f None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:47.722026 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b5199-b9af-4255-9f16-6b8e9e80055f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:47.722303 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b5199-b9af-4255-9f16-6b8e9e80055f None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:47.723031 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8b5199-b9af-4255-9f16-6b8e9e80055f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:57:47.723340 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8b5199-b9af-4255-9f16-6b8e9e80055f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:47.724710 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8b5199-b9af-4255-9f16-6b8e9e80055f None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:47.738717 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 610/1225] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:57:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:57:47.742524 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac91c79a-e0bd-4b4a-9847-fdf946433972 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:47.845199 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:47.851641 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac91c79a-e0bd-4b4a-9847-fdf946433972 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:47.893766 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc5433-e7cc-482b-9730-f83e79601df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:47.894405 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc5433-e7cc-482b-9730-f83e79601df8 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:47.894686 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc5433-e7cc-482b-9730-f83e79601df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:47.894999 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc5433-e7cc-482b-9730-f83e79601df8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:47.902772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dc5433-e7cc-482b-9730-f83e79601df8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:47.904275 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dc5433-e7cc-482b-9730-f83e79601df8 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:47.915292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-05dc5433-e7cc-482b-9730-f83e79601df8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 03 04:57:47.915292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:57:47.915292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:57:47.915292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:57:47.915292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:57:47.915292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:57:47.915292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:57:47.915292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:57:47.915292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:57:47.915292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:57:47.915292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:57:47.915292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:57:47.915292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:57:47.915292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:57:47.915292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:57:47.915292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:57:47.916433 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:57:47.916433 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:57:47.916433 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:57:47.916433 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:57:47.916433 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:57:47.916433 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:57:47.916433 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:57:47.916433 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:57:47.916433 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:57:47.916433 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:57:47.916433 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:57:47.916433 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:57:47.916433 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:57:47.916433 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:57:47.916433 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:57:47.916433 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:57:47.916433 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:57:47.916433 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:57:47.917693 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:57:47.917693 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:57:47.917693 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:57:47.917693 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:57:47.917693 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:57:47.917693 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:57:47.917693 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:57:47.917693 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:57:47.917693 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 03 04:57:47.917693 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:57:47.917693 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 616/1226] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:57:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:57:47.922950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b5199-b9af-4255-9f16-6b8e9e80055f None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:47.970649 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:48.023754 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b5199-b9af-4255-9f16-6b8e9e80055f None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:48.057721 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41034e6-762d-425d-8fba-3834e9e8ce1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:48.058330 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41034e6-762d-425d-8fba-3834e9e8ce1c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:48.058609 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41034e6-762d-425d-8fba-3834e9e8ce1c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:48.059027 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41034e6-762d-425d-8fba-3834e9e8ce1c None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:48.059813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41034e6-762d-425d-8fba-3834e9e8ce1c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:57:48.060121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41034e6-762d-425d-8fba-3834e9e8ce1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:48.061547 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41034e6-762d-425d-8fba-3834e9e8ce1c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:48.078393 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 611/1227] 10.222.0.40 () {56 vars in 1143 bytes} [Wed Aug 3 04:57:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:57:48.084163 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc5433-e7cc-482b-9730-f83e79601df8 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:48.183954 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc5433-e7cc-482b-9730-f83e79601df8 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:48.216638 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1757f5b-0b14-458b-9ee5-edfc59b9e1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:48.217273 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1757f5b-0b14-458b-9ee5-edfc59b9e1dd None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:48.217557 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1757f5b-0b14-458b-9ee5-edfc59b9e1dd None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:48.217833 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1757f5b-0b14-458b-9ee5-edfc59b9e1dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:48.228951 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1757f5b-0b14-458b-9ee5-edfc59b9e1dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:48.230402 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1757f5b-0b14-458b-9ee5-edfc59b9e1dd None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:48.245500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-b1757f5b-0b14-458b-9ee5-edfc59b9e1dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:57:48.245500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:57:48.245500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:57:48.245500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:57:48.245500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:57:48.245500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:57:48.245500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:57:48.245500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:57:48.245500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:57:48.245500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:57:48.245500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:57:48.245500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:57:48.245500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:57:48.245500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:57:48.245500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:57:48.246801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:57:48.246801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:57:48.246801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:57:48.246801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:57:48.246801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:57:48.246801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:57:48.246801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:57:48.246801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:57:48.246801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:57:48.246801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:57:48.246801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:57:48.246801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:57:48.246801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:57:48.246801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:57:48.246801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:57:48.246801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:57:48.246801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:57:48.247924 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:57:48.247924 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:57:48.247924 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:57:48.247924 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:57:48.247924 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 617/1228] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:57:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:57:48.253848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41034e6-762d-425d-8fba-3834e9e8ce1c None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:48.352762 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41034e6-762d-425d-8fba-3834e9e8ce1c None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:48.356368 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:48.387152 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2370755-330a-4240-89ae-002f52ca9d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:48.387768 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2370755-330a-4240-89ae-002f52ca9d1b None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:48.388059 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2370755-330a-4240-89ae-002f52ca9d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:48.388348 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2370755-330a-4240-89ae-002f52ca9d1b None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:48.389268 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2370755-330a-4240-89ae-002f52ca9d1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:57:48.389637 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2370755-330a-4240-89ae-002f52ca9d1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:48.391867 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2370755-330a-4240-89ae-002f52ca9d1b None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:48.403407 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 612/1229] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:57:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:57:48.409134 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1757f5b-0b14-458b-9ee5-edfc59b9e1dd None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:48.419282 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:48.508844 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1757f5b-0b14-458b-9ee5-edfc59b9e1dd None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:48.542662 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b601d431-edc9-4641-b0ae-f188d8c5f43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:48.543330 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601d431-edc9-4641-b0ae-f188d8c5f43a None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:48.543616 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601d431-edc9-4641-b0ae-f188d8c5f43a None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:48.543893 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601d431-edc9-4641-b0ae-f188d8c5f43a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:48.550517 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b601d431-edc9-4641-b0ae-f188d8c5f43a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:48.552008 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b601d431-edc9-4641-b0ae-f188d8c5f43a None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:48.563738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-b601d431-edc9-4641-b0ae-f188d8c5f43a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:57:48.563738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:57:48.563738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:57:48.563738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:57:48.563738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:57:48.563738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:57:48.563738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:57:48.563738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:57:48.563738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:57:48.563738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:57:48.563738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:57:48.563738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:57:48.563738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:57:48.563738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:57:48.563738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:57:48.564860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:57:48.564860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:57:48.564860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:57:48.564860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:57:48.564860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:57:48.564860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:57:48.564860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:57:48.564860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:57:48.564860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:57:48.564860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:57:48.564860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:57:48.564860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:57:48.564860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:57:48.564860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:57:48.564860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:57:48.564860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:57:48.564860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:57:48.565812 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:57:48.565812 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:57:48.565812 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:57:48.565812 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:57:48.565812 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:57:48.565812 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:57:48.565812 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:57:48.565812 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:57:48.565812 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 618/1230] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:57:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:57:48.571003 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2370755-330a-4240-89ae-002f52ca9d1b None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:48.672039 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2370755-330a-4240-89ae-002f52ca9d1b None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:48.707333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de380b-def9-48d6-b788-5b34f5c6696f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:48.707951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de380b-def9-48d6-b788-5b34f5c6696f None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:48.708235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de380b-def9-48d6-b788-5b34f5c6696f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:48.708511 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de380b-def9-48d6-b788-5b34f5c6696f None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:48.709257 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de380b-def9-48d6-b788-5b34f5c6696f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:57:48.709568 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de380b-def9-48d6-b788-5b34f5c6696f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:48.711033 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de380b-def9-48d6-b788-5b34f5c6696f None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:48.722899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 613/1231] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:57:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:57:48.729079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b601d431-edc9-4641-b0ae-f188d8c5f43a None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:48.795792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:48.830091 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b601d431-edc9-4641-b0ae-f188d8c5f43a None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:48.863470 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869eb199-baeb-4eac-9390-2d8f43f1d0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:48.864129 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869eb199-baeb-4eac-9390-2d8f43f1d0e0 None admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:48.864415 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869eb199-baeb-4eac-9390-2d8f43f1d0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:48.864695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869eb199-baeb-4eac-9390-2d8f43f1d0e0 None admin] PATH_INFO: `/v3/projects/cf3763a37383456184a9ca9351b4ce03/users/59846d88f4314c2382aa5cb8bae5053b/roles/cae571342db14f5fac7849efe7f8636e` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:48.879082 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869eb199-baeb-4eac-9390-2d8f43f1d0e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf3763a37383456184a9ca9351b4ce03, user_id=59846d88f4314c2382aa5cb8bae5053b, role_id=cae571342db14f5fac7849efe7f8636e)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:48.880535 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869eb199-baeb-4eac-9390-2d8f43f1d0e0 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:48.955675 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 619/1232] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 3 04:57:48 2022] PUT /identity/v3/projects/cf3763a37383456184a9ca9351b4ce03/users/59846d88f4314c2382aa5cb8bae5053b/roles/cae571342db14f5fac7849efe7f8636e => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 04:57:50.985854 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71b4ba-88ca-45bc-9572-6ff9669ae81a None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:50.986366 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71b4ba-88ca-45bc-9572-6ff9669ae81a None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:50.986366 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71b4ba-88ca-45bc-9572-6ff9669ae81a None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:50.987719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 614/1233] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:57:50.993166 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96487ae1-8707-4e7e-b3e7-c231b5bb5249 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:50.993624 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96487ae1-8707-4e7e-b3e7-c231b5bb5249 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:50.994293 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96487ae1-8707-4e7e-b3e7-c231b5bb5249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:51.012497 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-96487ae1-8707-4e7e-b3e7-c231b5bb5249 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:57:51.013691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-96487ae1-8707-4e7e-b3e7-c231b5bb5249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:57:51.079451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 620/1234] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:57:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:57:51.102389 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71b4ba-88ca-45bc-9572-6ff9669ae81a None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:51.128113 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:51.202878 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71b4ba-88ca-45bc-9572-6ff9669ae81a None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:51.236074 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134e0dd-98f3-4a6b-b411-ff52ad9ab969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:51.236642 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134e0dd-98f3-4a6b-b411-ff52ad9ab969 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:51.236922 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134e0dd-98f3-4a6b-b411-ff52ad9ab969 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:51.237196 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134e0dd-98f3-4a6b-b411-ff52ad9ab969 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:51.247537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7134e0dd-98f3-4a6b-b411-ff52ad9ab969 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:57:51.252327 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-7134e0dd-98f3-4a6b-b411-ff52ad9ab969 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:57:51.252327 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:57:51.252327 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:57:51.252327 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:57:51.252327 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:57:51.252327 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:57:51.252327 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:57:51.252327 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:57:51.252327 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:57:51.252327 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:57:51.252327 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:57:51.252327 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:57:51.252327 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:57:51.252327 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:57:51.252327 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:57:51.252327 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:57:51.253447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:57:51.253447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:57:51.253447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:57:51.253447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:57:51.253447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:57:51.253447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:57:51.253447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:57:51.253447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:57:51.253447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:57:51.253447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:57:51.253447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:57:51.253447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:57:51.253447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:57:51.253447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:57:51.253447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:57:51.253447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:57:51.254304 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:57:51.254304 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:57:51.254304 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:57:51.254304 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:57:51.254304 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:57:51.254304 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:57:51.254304 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:57:51.254304 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:57:51.254304 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:57:51.254304 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:57:51.254304 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 04:57:51.254304 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:57:51.254304 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 615/1235] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Aug 3 04:57:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:57:51.261546 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96487ae1-8707-4e7e-b3e7-c231b5bb5249 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:51.308992 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:51.371598 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96487ae1-8707-4e7e-b3e7-c231b5bb5249 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:51.405806 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24778f91-ff1d-4fc1-a528-30a75c50c0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:51.406422 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24778f91-ff1d-4fc1-a528-30a75c50c0d8 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:51.406700 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24778f91-ff1d-4fc1-a528-30a75c50c0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:51.407003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24778f91-ff1d-4fc1-a528-30a75c50c0d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:51.407718 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24778f91-ff1d-4fc1-a528-30a75c50c0d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:57:51.408020 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24778f91-ff1d-4fc1-a528-30a75c50c0d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:51.409405 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24778f91-ff1d-4fc1-a528-30a75c50c0d8 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:51.417764 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 621/1236] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Aug 3 04:57:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:57:51.423618 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134e0dd-98f3-4a6b-b411-ff52ad9ab969 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:51.509802 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:51.525970 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134e0dd-98f3-4a6b-b411-ff52ad9ab969 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:51.559416 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81199c-9696-48ef-a4c8-265a6836a166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:51.560035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81199c-9696-48ef-a4c8-265a6836a166 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:51.560315 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81199c-9696-48ef-a4c8-265a6836a166 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:51.560589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81199c-9696-48ef-a4c8-265a6836a166 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:51.570748 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c81199c-9696-48ef-a4c8-265a6836a166 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:51.572273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c81199c-9696-48ef-a4c8-265a6836a166 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:51.586426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-4c81199c-9696-48ef-a4c8-265a6836a166 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:57:51.586426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:57:51.586426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:57:51.586426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:57:51.586426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:57:51.586426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:57:51.586426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:57:51.586426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:57:51.586426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:57:51.586426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:57:51.586426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:57:51.586426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:57:51.586426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:57:51.586426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:57:51.586426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:57:51.592131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:57:51.592131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:57:51.592131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:57:51.592131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:57:51.592131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:57:51.592131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:57:51.592131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:57:51.592131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:57:51.592131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:57:51.592131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:57:51.592131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:57:51.592131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:57:51.592131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:57:51.592131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:57:51.592131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:57:51.592131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:57:51.592131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:57:51.593159 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:57:51.593159 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:57:51.593159 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:57:51.593159 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:57:51.593159 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 616/1237] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:57:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:57:51.595010 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24778f91-ff1d-4fc1-a528-30a75c50c0d8 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:51.695043 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24778f91-ff1d-4fc1-a528-30a75c50c0d8 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:51.703981 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:51.729544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5753033c-3fb1-4d5d-8f5f-48675548bfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:51.733531 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753033c-3fb1-4d5d-8f5f-48675548bfe2 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:51.734495 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753033c-3fb1-4d5d-8f5f-48675548bfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:51.734495 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753033c-3fb1-4d5d-8f5f-48675548bfe2 None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:51.735289 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5753033c-3fb1-4d5d-8f5f-48675548bfe2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:57:51.735838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5753033c-3fb1-4d5d-8f5f-48675548bfe2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:51.737250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5753033c-3fb1-4d5d-8f5f-48675548bfe2 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:51.752077 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 622/1238] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:57:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:57:51.757883 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81199c-9696-48ef-a4c8-265a6836a166 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:51.864554 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81199c-9696-48ef-a4c8-265a6836a166 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:51.898473 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8784e6f-0259-44fd-8d84-e3807f48d3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:51.899148 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8784e6f-0259-44fd-8d84-e3807f48d3fc None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:51.899431 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8784e6f-0259-44fd-8d84-e3807f48d3fc None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:51.899706 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8784e6f-0259-44fd-8d84-e3807f48d3fc None admin] PATH_INFO: `/v3/users/placement` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:51.903356 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:51.908381 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8784e6f-0259-44fd-8d84-e3807f48d3fc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:51.909826 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8784e6f-0259-44fd-8d84-e3807f48d3fc None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:51.921025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-b8784e6f-0259-44fd-8d84-e3807f48d3fc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 03 04:57:51.921025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:57:51.921025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:57:51.921025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:57:51.921025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:57:51.921025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:57:51.921025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:57:51.921025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:57:51.921025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:57:51.921025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:57:51.921025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:57:51.921025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:57:51.921025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:57:51.921025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:57:51.921025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:57:51.921025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:57:51.922194 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:57:51.922194 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:57:51.922194 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:57:51.922194 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:57:51.922194 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:57:51.922194 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:57:51.922194 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:57:51.922194 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:57:51.922194 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:57:51.922194 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:57:51.922194 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:57:51.922194 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:57:51.922194 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:57:51.922194 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:57:51.922194 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:57:51.922194 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:57:51.922194 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:57:51.922194 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:57:51.923427 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:57:51.923427 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:57:51.923427 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:57:51.923427 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:57:51.923427 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:57:51.923427 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:57:51.923427 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:57:51.923427 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:57:51.923427 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 03 04:57:51.923427 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:57:51.923427 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 617/1239] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:57:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:57:51.928219 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5753033c-3fb1-4d5d-8f5f-48675548bfe2 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:52.029682 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5753033c-3fb1-4d5d-8f5f-48675548bfe2 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:52.064070 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bae37f-b2f4-4c96-ba2f-45ef9ea46576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:52.064681 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bae37f-b2f4-4c96-ba2f-45ef9ea46576 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:52.064966 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bae37f-b2f4-4c96-ba2f-45ef9ea46576 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:52.065244 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bae37f-b2f4-4c96-ba2f-45ef9ea46576 None admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:52.066029 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bae37f-b2f4-4c96-ba2f-45ef9ea46576 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:57:52.066366 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bae37f-b2f4-4c96-ba2f-45ef9ea46576 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:52.067824 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bae37f-b2f4-4c96-ba2f-45ef9ea46576 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:52.085487 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 623/1240] 10.222.0.40 () {56 vars in 1143 bytes} [Wed Aug 3 04:57:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:57:52.092018 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8784e6f-0259-44fd-8d84-e3807f48d3fc None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:52.193936 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8784e6f-0259-44fd-8d84-e3807f48d3fc None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:52.227146 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd78a79d-bba8-4702-956d-c4eeb66353a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:52.227756 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78a79d-bba8-4702-956d-c4eeb66353a3 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:52.228039 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78a79d-bba8-4702-956d-c4eeb66353a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:52.228313 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78a79d-bba8-4702-956d-c4eeb66353a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:52.292487 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:52.299082 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd78a79d-bba8-4702-956d-c4eeb66353a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:52.300553 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd78a79d-bba8-4702-956d-c4eeb66353a3 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:52.404659 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 618/1241] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:57:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 04:57:52.410526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bae37f-b2f4-4c96-ba2f-45ef9ea46576 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:52.437920 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:52.511510 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bae37f-b2f4-4c96-ba2f-45ef9ea46576 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:52.550077 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e372ed1-d530-4b5c-80bd-28667a0cb8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:52.550710 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e372ed1-d530-4b5c-80bd-28667a0cb8fb None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:52.551021 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e372ed1-d530-4b5c-80bd-28667a0cb8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:52.551300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e372ed1-d530-4b5c-80bd-28667a0cb8fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:52.561776 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e372ed1-d530-4b5c-80bd-28667a0cb8fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:52.563276 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e372ed1-d530-4b5c-80bd-28667a0cb8fb None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:52.577655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-7e372ed1-d530-4b5c-80bd-28667a0cb8fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:57:52.577655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:57:52.577655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:57:52.577655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:57:52.577655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:57:52.577655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:57:52.577655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:57:52.577655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:57:52.577655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:57:52.577655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:57:52.577655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:57:52.577655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:57:52.577655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:57:52.577655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:57:52.577655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:57:52.580527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:57:52.580527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:57:52.580527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:57:52.580527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:57:52.580527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:57:52.580527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:57:52.580527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:57:52.580527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:57:52.580527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:57:52.580527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:57:52.580527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:57:52.580527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:57:52.580527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:57:52.580527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:57:52.580527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:57:52.580527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:57:52.580527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:57:52.581448 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:57:52.581448 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:57:52.581448 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:57:52.581448 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:57:52.581448 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 624/1242] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:57:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:57:52.584544 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd78a79d-bba8-4702-956d-c4eeb66353a3 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:52.677769 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:52.684964 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd78a79d-bba8-4702-956d-c4eeb66353a3 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:52.748553 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6622fe7-3816-4309-bb32-54e968dcebf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:52.748553 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6622fe7-3816-4309-bb32-54e968dcebf2 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:52.748553 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6622fe7-3816-4309-bb32-54e968dcebf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:52.748553 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6622fe7-3816-4309-bb32-54e968dcebf2 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:52.748975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6622fe7-3816-4309-bb32-54e968dcebf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:57:52.748975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6622fe7-3816-4309-bb32-54e968dcebf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:52.748975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6622fe7-3816-4309-bb32-54e968dcebf2 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:52.748975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 619/1243] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:57:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:57:52.749307 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e372ed1-d530-4b5c-80bd-28667a0cb8fb None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:52.824131 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:52.845463 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e372ed1-d530-4b5c-80bd-28667a0cb8fb None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:52.879374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e22964-af83-4558-978e-b1a95bf4fadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:52.880011 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e22964-af83-4558-978e-b1a95bf4fadd None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:52.880293 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e22964-af83-4558-978e-b1a95bf4fadd None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:52.880566 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e22964-af83-4558-978e-b1a95bf4fadd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:52.887187 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e22964-af83-4558-978e-b1a95bf4fadd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:52.888633 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e22964-af83-4558-978e-b1a95bf4fadd None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:52.900957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-a5e22964-af83-4558-978e-b1a95bf4fadd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:57:52.900957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:57:52.900957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:57:52.900957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:57:52.900957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:57:52.900957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:57:52.900957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:57:52.900957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:57:52.900957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:57:52.900957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:57:52.900957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:57:52.900957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:57:52.900957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:57:52.900957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:57:52.900957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:57:52.901965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:57:52.901965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:57:52.901965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:57:52.901965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:57:52.901965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:57:52.901965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:57:52.901965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:57:52.901965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:57:52.901965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:57:52.901965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:57:52.901965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:57:52.901965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:57:52.901965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:57:52.901965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:57:52.901965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:57:52.901965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:57:52.901965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:57:52.903159 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:57:52.903159 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:57:52.903159 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:57:52.903159 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:57:52.903159 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:57:52.903159 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:57:52.903159 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:57:52.903159 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:57:52.903159 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 625/1244] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:57:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:57:52.908024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6622fe7-3816-4309-bb32-54e968dcebf2 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:53.006373 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6622fe7-3816-4309-bb32-54e968dcebf2 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:53.041310 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a778dc11-c6a1-434a-a10d-591349c77850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:53.041928 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778dc11-c6a1-434a-a10d-591349c77850 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:53.042214 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778dc11-c6a1-434a-a10d-591349c77850 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:53.042493 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778dc11-c6a1-434a-a10d-591349c77850 None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:53.043276 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a778dc11-c6a1-434a-a10d-591349c77850 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:57:53.043588 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a778dc11-c6a1-434a-a10d-591349c77850 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:53.045024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a778dc11-c6a1-434a-a10d-591349c77850 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:53.057008 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 620/1245] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:57:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 04:57:53.063354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e22964-af83-4558-978e-b1a95bf4fadd None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:53.167599 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e22964-af83-4558-978e-b1a95bf4fadd None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:53.200444 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d051847a-331a-43fd-b55b-3ed5936af86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:53.201053 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051847a-331a-43fd-b55b-3ed5936af86e None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:53.201332 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051847a-331a-43fd-b55b-3ed5936af86e None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:53.201601 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051847a-331a-43fd-b55b-3ed5936af86e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:53.202378 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d051847a-331a-43fd-b55b-3ed5936af86e None admin] RBAC: Adding query filter params (role.id=cae571342db14f5fac7849efe7f8636e, scope.project.id=cf3763a37383456184a9ca9351b4ce03, user.id=59846d88f4314c2382aa5cb8bae5053b) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:57:53.202682 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d051847a-331a-43fd-b55b-3ed5936af86e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:53.204196 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d051847a-331a-43fd-b55b-3ed5936af86e None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:53.210050 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 626/1246] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:57:53 2022] GET /identity/v3/role_assignments?user.id=59846d88f4314c2382aa5cb8bae5053b&scope.project.id=cf3763a37383456184a9ca9351b4ce03&role.id=cae571342db14f5fac7849efe7f8636e => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:57:55.310226 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21e4fe-2994-4e33-b412-4a090e4b81fc None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:55.310752 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21e4fe-2994-4e33-b412-4a090e4b81fc None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:55.310887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21e4fe-2994-4e33-b412-4a090e4b81fc None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:55.312181 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 621/1247] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:57:55.317551 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42157c13-f644-40b2-a4e7-37c7913b1cca None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:55.318007 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42157c13-f644-40b2-a4e7-37c7913b1cca None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:55.318112 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42157c13-f644-40b2-a4e7-37c7913b1cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:55.324617 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:55.337087 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-42157c13-f644-40b2-a4e7-37c7913b1cca None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:57:55.338826 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-42157c13-f644-40b2-a4e7-37c7913b1cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:57:55.405819 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 627/1248] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:57:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:57:55.429655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21e4fe-2994-4e33-b412-4a090e4b81fc None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:55.447750 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:55.536960 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21e4fe-2994-4e33-b412-4a090e4b81fc None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:55.571700 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0019b-2805-4735-b3e0-8542433d9336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:55.572334 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0019b-2805-4735-b3e0-8542433d9336 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:55.572618 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0019b-2805-4735-b3e0-8542433d9336 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:55.572895 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0019b-2805-4735-b3e0-8542433d9336 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:55.582908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0019b-2805-4735-b3e0-8542433d9336 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:57:55.587901 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-31b0019b-2805-4735-b3e0-8542433d9336 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:57:55.587901 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:57:55.587901 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:57:55.587901 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:57:55.587901 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:57:55.587901 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:57:55.587901 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:57:55.587901 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:57:55.587901 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:57:55.587901 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:57:55.587901 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:57:55.587901 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:57:55.587901 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:57:55.587901 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:57:55.587901 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:57:55.587901 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:57:55.589066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:57:55.589066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:57:55.589066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:57:55.589066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:57:55.589066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:57:55.589066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:57:55.589066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:57:55.589066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:57:55.589066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:57:55.589066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:57:55.589066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:57:55.589066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:57:55.589066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:57:55.589066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:57:55.589066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:57:55.589066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:57:55.590186 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:57:55.590186 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:57:55.590186 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:57:55.590186 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:57:55.590186 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:57:55.590186 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:57:55.590186 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:57:55.590186 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:57:55.590186 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:57:55.590186 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:57:55.590186 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:57:55.590186 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:57:55.590186 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 622/1249] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:57:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:57:55.595295 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42157c13-f644-40b2-a4e7-37c7913b1cca None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:55.695291 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42157c13-f644-40b2-a4e7-37c7913b1cca None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:55.723622 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:55.729078 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ae419-556f-49cb-b654-77f3fc437210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:55.729693 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ae419-556f-49cb-b654-77f3fc437210 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:55.729978 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ae419-556f-49cb-b654-77f3fc437210 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:55.730255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ae419-556f-49cb-b654-77f3fc437210 None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:55.730997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ae419-556f-49cb-b654-77f3fc437210 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:57:55.731379 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ae419-556f-49cb-b654-77f3fc437210 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:55.732742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ae419-556f-49cb-b654-77f3fc437210 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:55.740579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 628/1250] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:57:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:57:55.747032 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0019b-2805-4735-b3e0-8542433d9336 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:55.826128 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:55.853956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0019b-2805-4735-b3e0-8542433d9336 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:55.891184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b7755-cb85-448b-b0c5-537ff636444b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:55.891816 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b7755-cb85-448b-b0c5-537ff636444b None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:55.892098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b7755-cb85-448b-b0c5-537ff636444b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:55.892369 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b7755-cb85-448b-b0c5-537ff636444b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:55.902860 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b7755-cb85-448b-b0c5-537ff636444b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:55.904428 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b7755-cb85-448b-b0c5-537ff636444b None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:55.920134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-701b7755-cb85-448b-b0c5-537ff636444b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:57:55.920134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:57:55.920134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:57:55.920134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:57:55.920134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:57:55.920134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:57:55.920134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:57:55.920134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:57:55.920134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:57:55.920134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:57:55.920134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:57:55.920134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:57:55.920134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:57:55.920134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:57:55.920134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:57:55.921905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:57:55.921905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:57:55.921905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:57:55.921905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:57:55.921905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:57:55.921905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:57:55.921905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:57:55.921905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:57:55.921905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:57:55.921905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:57:55.921905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:57:55.921905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:57:55.921905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:57:55.921905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:57:55.921905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:57:55.921905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:57:55.921905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:57:55.922833 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:57:55.922833 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:57:55.922833 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:57:55.922833 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:57:55.922833 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 623/1251] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:57:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:57:55.927200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ae419-556f-49cb-b654-77f3fc437210 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:56.033710 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ae419-556f-49cb-b654-77f3fc437210 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:56.068801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be1392-4972-459d-ada9-ba96a009c8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:56.069424 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be1392-4972-459d-ada9-ba96a009c8a4 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:56.069706 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be1392-4972-459d-ada9-ba96a009c8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:56.069978 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be1392-4972-459d-ada9-ba96a009c8a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:56.070668 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be1392-4972-459d-ada9-ba96a009c8a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:57:56.071014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be1392-4972-459d-ada9-ba96a009c8a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:56.072520 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be1392-4972-459d-ada9-ba96a009c8a4 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:56.084224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 629/1252] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:57:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:57:56.090429 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b7755-cb85-448b-b0c5-537ff636444b None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:56.189880 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b7755-cb85-448b-b0c5-537ff636444b None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:56.215548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:56.223064 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e092a9-bc2d-44cf-ba2f-4c76d43d1396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:56.223698 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e092a9-bc2d-44cf-ba2f-4c76d43d1396 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:56.223974 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e092a9-bc2d-44cf-ba2f-4c76d43d1396 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:56.224246 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e092a9-bc2d-44cf-ba2f-4c76d43d1396 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:56.232131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e092a9-bc2d-44cf-ba2f-4c76d43d1396 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:56.237431 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e092a9-bc2d-44cf-ba2f-4c76d43d1396 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:56.245099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-c3e092a9-bc2d-44cf-ba2f-4c76d43d1396 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 03 04:57:56.245099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:57:56.245099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:57:56.245099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:57:56.245099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:57:56.245099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:57:56.245099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:57:56.245099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:57:56.245099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:57:56.245099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:57:56.245099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:57:56.245099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:57:56.245099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:57:56.245099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:57:56.245099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:57:56.245099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:57:56.246270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:57:56.246270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:57:56.246270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:57:56.246270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:57:56.246270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:57:56.246270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:57:56.246270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:57:56.246270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:57:56.246270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:57:56.246270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:57:56.246270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:57:56.246270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:57:56.246270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:57:56.246270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:57:56.246270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:57:56.246270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:57:56.246270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:57:56.246270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:57:56.247512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:57:56.247512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:57:56.247512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:57:56.247512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:57:56.247512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:57:56.247512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:57:56.247512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:57:56.247512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:57:56.247512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 03 04:57:56.247512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:57:56.247512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 624/1253] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:57:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:57:56.252034 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be1392-4972-459d-ada9-ba96a009c8a4 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:56.308260 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:56.352163 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be1392-4972-459d-ada9-ba96a009c8a4 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:56.387247 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e400c0-dd01-41ef-80fe-9647bc2e4961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:56.387856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e400c0-dd01-41ef-80fe-9647bc2e4961 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:56.388133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e400c0-dd01-41ef-80fe-9647bc2e4961 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:56.388405 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e400c0-dd01-41ef-80fe-9647bc2e4961 None admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:56.389180 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e400c0-dd01-41ef-80fe-9647bc2e4961 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:57:56.389482 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e400c0-dd01-41ef-80fe-9647bc2e4961 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:56.391186 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e400c0-dd01-41ef-80fe-9647bc2e4961 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:56.408078 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 630/1254] 10.222.0.40 () {56 vars in 1143 bytes} [Wed Aug 3 04:57:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:57:56.414131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e092a9-bc2d-44cf-ba2f-4c76d43d1396 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:56.512741 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e092a9-bc2d-44cf-ba2f-4c76d43d1396 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:56.546044 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8536bf-4eb3-4be4-8dca-30b9035f71dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:56.546650 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8536bf-4eb3-4be4-8dca-30b9035f71dc None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:56.546961 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8536bf-4eb3-4be4-8dca-30b9035f71dc None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:56.547239 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8536bf-4eb3-4be4-8dca-30b9035f71dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:56.603863 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:56.616129 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8536bf-4eb3-4be4-8dca-30b9035f71dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:56.617670 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8536bf-4eb3-4be4-8dca-30b9035f71dc None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:56.725875 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 625/1255] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:57:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 04:57:56.732457 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e400c0-dd01-41ef-80fe-9647bc2e4961 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:56.832514 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e400c0-dd01-41ef-80fe-9647bc2e4961 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:56.846486 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:56.868042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0310c-7cc0-4fd3-8421-94a7a75a0c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:56.868768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0310c-7cc0-4fd3-8421-94a7a75a0c43 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:56.869006 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0310c-7cc0-4fd3-8421-94a7a75a0c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:56.869278 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0310c-7cc0-4fd3-8421-94a7a75a0c43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:56.879910 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc0310c-7cc0-4fd3-8421-94a7a75a0c43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:56.881370 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc0310c-7cc0-4fd3-8421-94a7a75a0c43 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:56.895874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-ffc0310c-7cc0-4fd3-8421-94a7a75a0c43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:57:56.895874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:57:56.895874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:57:56.895874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:57:56.895874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:57:56.895874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:57:56.895874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:57:56.895874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:57:56.895874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:57:56.895874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:57:56.895874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:57:56.895874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:57:56.895874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:57:56.895874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:57:56.895874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:57:56.896928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:57:56.896928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:57:56.896928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:57:56.896928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:57:56.896928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:57:56.896928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:57:56.896928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:57:56.896928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:57:56.896928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:57:56.896928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:57:56.896928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:57:56.896928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:57:56.896928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:57:56.896928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:57:56.896928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:57:56.896928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:57:56.896928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:57:56.897965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:57:56.897965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:57:56.897965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:57:56.897965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:57:56.897965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 631/1256] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:57:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:57:56.903678 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8536bf-4eb3-4be4-8dca-30b9035f71dc None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:56.992655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:57.004446 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8536bf-4eb3-4be4-8dca-30b9035f71dc None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:57.040514 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f800eb-4105-4587-99e2-46dcfd55dfa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:57.041049 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f800eb-4105-4587-99e2-46dcfd55dfa3 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:57.041333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f800eb-4105-4587-99e2-46dcfd55dfa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:57.041610 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f800eb-4105-4587-99e2-46dcfd55dfa3 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:57.042292 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f800eb-4105-4587-99e2-46dcfd55dfa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:57:57.042599 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f800eb-4105-4587-99e2-46dcfd55dfa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:57.044003 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f800eb-4105-4587-99e2-46dcfd55dfa3 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:57.055244 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 626/1257] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:57:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:57:57.060939 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0310c-7cc0-4fd3-8421-94a7a75a0c43 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:57.161553 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0310c-7cc0-4fd3-8421-94a7a75a0c43 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:57.193742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1119405-6b0f-43dd-af76-5b5f72b54404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:57.194368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1119405-6b0f-43dd-af76-5b5f72b54404 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:57.194646 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1119405-6b0f-43dd-af76-5b5f72b54404 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:57.194944 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1119405-6b0f-43dd-af76-5b5f72b54404 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:57.202286 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1119405-6b0f-43dd-af76-5b5f72b54404 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:57.203761 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1119405-6b0f-43dd-af76-5b5f72b54404 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:57.215534 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-e1119405-6b0f-43dd-af76-5b5f72b54404 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:57:57.215534 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:57:57.215534 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:57:57.215534 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:57:57.215534 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:57:57.215534 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:57:57.215534 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:57:57.215534 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:57:57.215534 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:57:57.215534 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:57:57.215534 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:57:57.215534 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:57:57.215534 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:57:57.215534 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:57:57.215534 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:57:57.216570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:57:57.216570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:57:57.216570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:57:57.216570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:57:57.216570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:57:57.216570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:57:57.216570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:57:57.216570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:57:57.216570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:57:57.216570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:57:57.216570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:57:57.216570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:57:57.216570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:57:57.216570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:57:57.216570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:57:57.216570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:57:57.216570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:57:57.217740 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:57:57.217740 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:57:57.217740 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:57:57.217740 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:57:57.217740 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:57:57.217740 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:57:57.217740 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:57:57.217740 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:57:57.217740 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 632/1258] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:57:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:57:57.222650 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f800eb-4105-4587-99e2-46dcfd55dfa3 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:57.323318 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f800eb-4105-4587-99e2-46dcfd55dfa3 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:57.356131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6874648f-ac46-469c-9314-f34a0a500466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:57.356756 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874648f-ac46-469c-9314-f34a0a500466 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:57.357035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874648f-ac46-469c-9314-f34a0a500466 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:57.357304 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874648f-ac46-469c-9314-f34a0a500466 None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:57.358058 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6874648f-ac46-469c-9314-f34a0a500466 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:57:57.358348 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6874648f-ac46-469c-9314-f34a0a500466 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:57.359784 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6874648f-ac46-469c-9314-f34a0a500466 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:57.371627 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 627/1259] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:57:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:57:57.378232 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1119405-6b0f-43dd-af76-5b5f72b54404 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:57.406537 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:57.478790 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1119405-6b0f-43dd-af76-5b5f72b54404 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:57.511768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f735f-49ca-463e-a9d3-395123d98163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:57.512404 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f735f-49ca-463e-a9d3-395123d98163 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:57.512684 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f735f-49ca-463e-a9d3-395123d98163 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:57.512956 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f735f-49ca-463e-a9d3-395123d98163 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:57.513732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f735f-49ca-463e-a9d3-395123d98163 None admin] RBAC: Adding query filter params (role.id=e39d4ecea4c64d5794e1079c9be3ad36, scope.project.id=cf3763a37383456184a9ca9351b4ce03, user.id=59846d88f4314c2382aa5cb8bae5053b) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:57:57.514040 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f735f-49ca-463e-a9d3-395123d98163 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:57:57.515792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f735f-49ca-463e-a9d3-395123d98163 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:57:57.521032 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 633/1260] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:57:57 2022] GET /identity/v3/role_assignments?user.id=59846d88f4314c2382aa5cb8bae5053b&scope.project.id=cf3763a37383456184a9ca9351b4ce03&role.id=e39d4ecea4c64d5794e1079c9be3ad36 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:57:59.580802 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e879e62-3a50-45c1-b8bf-9fb8b2674bf6 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:59.581517 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e879e62-3a50-45c1-b8bf-9fb8b2674bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:59.581517 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e879e62-3a50-45c1-b8bf-9fb8b2674bf6 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:59.582744 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 628/1261] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:57:59.588616 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fcfd7a-674e-41ad-b0cc-ec3bbfa8e1ac None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:59.589090 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fcfd7a-674e-41ad-b0cc-ec3bbfa8e1ac None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:59.589090 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fcfd7a-674e-41ad-b0cc-ec3bbfa8e1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:59.607881 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-18fcfd7a-674e-41ad-b0cc-ec3bbfa8e1ac None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:57:59.609093 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-18fcfd7a-674e-41ad-b0cc-ec3bbfa8e1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:57:59.698860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 634/1262] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:57:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:57:59.723277 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e879e62-3a50-45c1-b8bf-9fb8b2674bf6 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:59.737458 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:59.834345 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e879e62-3a50-45c1-b8bf-9fb8b2674bf6 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:57:59.867912 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d67642-37f7-4735-ab2c-a7579fc8adc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:57:59.869048 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d67642-37f7-4735-ab2c-a7579fc8adc6 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:57:59.869333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d67642-37f7-4735-ab2c-a7579fc8adc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:57:59.869611 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d67642-37f7-4735-ab2c-a7579fc8adc6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:57:59.879560 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d67642-37f7-4735-ab2c-a7579fc8adc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:57:59.885041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-33d67642-37f7-4735-ab2c-a7579fc8adc6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:57:59.885041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:57:59.885041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:57:59.885041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:57:59.885041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:57:59.885041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:57:59.885041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:57:59.885041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:57:59.885041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:57:59.885041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:57:59.885041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:57:59.885041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:57:59.885041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:57:59.885041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:57:59.885041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:57:59.885041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:57:59.886061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:57:59.886061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:57:59.886061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:57:59.886061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:57:59.886061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:57:59.886061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:57:59.886061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:57:59.886061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:57:59.886061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:57:59.886061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:57:59.886061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:57:59.886061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:57:59.886061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:57:59.886061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:57:59.886061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:57:59.886061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:57:59.887262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:57:59.887262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:57:59.887262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:57:59.887262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:57:59.887262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:57:59.887262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:57:59.887262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:57:59.887262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:57:59.887262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:57:59.887262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:57:59.887262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:57:59.887262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:57:59.887262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 629/1263] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:57:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:57:59.892622 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fcfd7a-674e-41ad-b0cc-ec3bbfa8e1ac None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:57:59.915308 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:57:59.997807 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fcfd7a-674e-41ad-b0cc-ec3bbfa8e1ac None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:00.033006 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633fc8c-d74a-496d-bd4a-ca483db207ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:00.033632 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633fc8c-d74a-496d-bd4a-ca483db207ee None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:00.033915 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633fc8c-d74a-496d-bd4a-ca483db207ee None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:00.034192 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633fc8c-d74a-496d-bd4a-ca483db207ee None admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:00.034936 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633fc8c-d74a-496d-bd4a-ca483db207ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:58:00.035248 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633fc8c-d74a-496d-bd4a-ca483db207ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:00.036627 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633fc8c-d74a-496d-bd4a-ca483db207ee None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:00.045282 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 635/1264] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:57:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:58:00.051117 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d67642-37f7-4735-ab2c-a7579fc8adc6 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:00.118793 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:58:00.151210 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d67642-37f7-4735-ab2c-a7579fc8adc6 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:00.184488 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a48f22-90bc-4ef7-b1c6-08bd5dfe1e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:00.185125 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a48f22-90bc-4ef7-b1c6-08bd5dfe1e0f None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:00.185408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a48f22-90bc-4ef7-b1c6-08bd5dfe1e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:00.185682 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a48f22-90bc-4ef7-b1c6-08bd5dfe1e0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:00.197072 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a48f22-90bc-4ef7-b1c6-08bd5dfe1e0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:00.198624 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a48f22-90bc-4ef7-b1c6-08bd5dfe1e0f None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:00.213720 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-29a48f22-90bc-4ef7-b1c6-08bd5dfe1e0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:58:00.213720 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:58:00.213720 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:58:00.213720 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:58:00.213720 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:58:00.213720 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:58:00.213720 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:58:00.213720 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:58:00.213720 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:58:00.213720 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:58:00.213720 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:58:00.213720 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:58:00.213720 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:58:00.213720 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:58:00.213720 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:58:00.214832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:58:00.214832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:58:00.214832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:58:00.214832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:58:00.214832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:58:00.214832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:58:00.214832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:58:00.214832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:58:00.214832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:58:00.214832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:58:00.214832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:58:00.214832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:58:00.214832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:58:00.214832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:58:00.214832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:58:00.214832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:58:00.214832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:58:00.216009 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:58:00.216009 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:58:00.216009 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:58:00.216009 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:58:00.216009 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 630/1265] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:58:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:58:00.220596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633fc8c-d74a-496d-bd4a-ca483db207ee None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:00.300997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:58:00.319796 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633fc8c-d74a-496d-bd4a-ca483db207ee None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:00.356302 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d0f88-4299-41ef-af7a-ddfd450767e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:00.356926 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d0f88-4299-41ef-af7a-ddfd450767e7 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:00.357214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d0f88-4299-41ef-af7a-ddfd450767e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:00.357509 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d0f88-4299-41ef-af7a-ddfd450767e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:00.358185 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d0f88-4299-41ef-af7a-ddfd450767e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:58:00.358493 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d0f88-4299-41ef-af7a-ddfd450767e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:00.359905 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d0f88-4299-41ef-af7a-ddfd450767e7 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:00.371695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 636/1266] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:58:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:58:00.377300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a48f22-90bc-4ef7-b1c6-08bd5dfe1e0f None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:00.477683 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a48f22-90bc-4ef7-b1c6-08bd5dfe1e0f None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:00.497242 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:58:00.511793 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa1bed-0a4e-450f-835e-311bf0c35911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:00.512429 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa1bed-0a4e-450f-835e-311bf0c35911 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:00.512715 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa1bed-0a4e-450f-835e-311bf0c35911 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:00.512996 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa1bed-0a4e-450f-835e-311bf0c35911 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:00.520608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa1bed-0a4e-450f-835e-311bf0c35911 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:00.522047 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa1bed-0a4e-450f-835e-311bf0c35911 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:00.533612 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-e0aa1bed-0a4e-450f-835e-311bf0c35911 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 03 04:58:00.533612 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:58:00.533612 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:58:00.533612 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:58:00.533612 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:58:00.533612 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:58:00.533612 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:58:00.533612 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:58:00.533612 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:58:00.533612 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:58:00.533612 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:58:00.533612 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:58:00.533612 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:58:00.533612 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:58:00.533612 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:58:00.533612 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:58:00.534770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:58:00.534770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:58:00.534770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:58:00.534770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:58:00.534770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:58:00.534770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:58:00.534770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:58:00.534770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:58:00.534770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:58:00.534770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:58:00.534770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:58:00.534770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:58:00.534770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:58:00.534770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:58:00.534770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:58:00.534770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:58:00.534770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:58:00.534770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:58:00.535985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:58:00.535985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:58:00.535985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:58:00.535985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:58:00.535985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:58:00.535985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:58:00.535985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:58:00.535985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:58:00.535985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 03 04:58:00.535985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:58:00.535985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 631/1267] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:58:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:58:00.540719 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d0f88-4299-41ef-af7a-ddfd450767e7 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:00.645729 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d0f88-4299-41ef-af7a-ddfd450767e7 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:00.682226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1d117-cbd6-416a-b4a6-0c0250ddf15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:00.682865 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1d117-cbd6-416a-b4a6-0c0250ddf15a None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:00.683165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1d117-cbd6-416a-b4a6-0c0250ddf15a None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:00.683443 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1d117-cbd6-416a-b4a6-0c0250ddf15a None admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:00.684229 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1d117-cbd6-416a-b4a6-0c0250ddf15a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:58:00.684538 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1d117-cbd6-416a-b4a6-0c0250ddf15a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:00.686106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1d117-cbd6-416a-b4a6-0c0250ddf15a None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:00.695127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:58:00.704419 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 637/1268] 10.222.0.40 () {56 vars in 1143 bytes} [Wed Aug 3 04:58:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:58:00.710094 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa1bed-0a4e-450f-835e-311bf0c35911 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:00.815236 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa1bed-0a4e-450f-835e-311bf0c35911 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:00.851201 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4387c-42b8-4fd8-ad88-23f288f0d845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:00.851847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4387c-42b8-4fd8-ad88-23f288f0d845 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:00.852127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4387c-42b8-4fd8-ad88-23f288f0d845 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:00.852979 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4387c-42b8-4fd8-ad88-23f288f0d845 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:00.863692 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f4387c-42b8-4fd8-ad88-23f288f0d845 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:00.865137 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f4387c-42b8-4fd8-ad88-23f288f0d845 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:00.882410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-31f4387c-42b8-4fd8-ad88-23f288f0d845 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:58:00.882410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:58:00.882410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:58:00.882410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:58:00.882410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:58:00.882410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:58:00.882410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:58:00.882410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:58:00.882410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:58:00.882410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:58:00.882410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:58:00.882410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:58:00.882410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:58:00.882410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:58:00.882410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:58:00.885273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:58:00.885273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:58:00.885273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:58:00.885273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:58:00.885273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:58:00.885273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:58:00.885273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:58:00.885273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:58:00.885273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:58:00.885273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:58:00.885273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:58:00.885273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:58:00.885273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:58:00.885273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:58:00.885273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:58:00.885273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:58:00.885273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:58:00.886375 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:58:00.886375 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:58:00.886375 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:58:00.886375 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:58:00.888337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 632/1269] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:58:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:58:00.889330 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1d117-cbd6-416a-b4a6-0c0250ddf15a None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:00.997234 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1d117-cbd6-416a-b4a6-0c0250ddf15a None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:01.032899 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aac15a-8efd-496b-a8a2-b2e591b159e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:01.033780 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aac15a-8efd-496b-a8a2-b2e591b159e9 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:01.034064 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aac15a-8efd-496b-a8a2-b2e591b159e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:01.034339 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aac15a-8efd-496b-a8a2-b2e591b159e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:01.035064 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aac15a-8efd-496b-a8a2-b2e591b159e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:58:01.035371 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aac15a-8efd-496b-a8a2-b2e591b159e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:01.036747 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aac15a-8efd-496b-a8a2-b2e591b159e9 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:01.049762 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 638/1270] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:58:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:58:01.055352 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4387c-42b8-4fd8-ad88-23f288f0d845 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:01.087622 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:58:01.157001 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4387c-42b8-4fd8-ad88-23f288f0d845 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:01.191118 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a81a6-1aa4-4baa-840a-b03610a5014e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:01.191752 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a81a6-1aa4-4baa-840a-b03610a5014e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:01.192032 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a81a6-1aa4-4baa-840a-b03610a5014e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:01.192304 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a81a6-1aa4-4baa-840a-b03610a5014e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:01.198863 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a81a6-1aa4-4baa-840a-b03610a5014e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:01.200353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a81a6-1aa4-4baa-840a-b03610a5014e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:01.211537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-ee9a81a6-1aa4-4baa-840a-b03610a5014e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:58:01.211537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:58:01.211537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:58:01.211537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:58:01.211537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:58:01.211537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:58:01.211537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:58:01.211537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:58:01.211537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:58:01.211537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:58:01.211537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:58:01.211537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:58:01.211537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:58:01.211537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:58:01.211537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:58:01.213121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:58:01.213121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:58:01.213121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:58:01.213121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:58:01.213121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:58:01.213121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:58:01.213121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:58:01.213121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:58:01.213121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:58:01.213121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:58:01.213121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:58:01.213121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:58:01.213121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:58:01.213121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:58:01.213121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:58:01.213121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:58:01.213121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:58:01.214475 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:58:01.214475 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:58:01.214475 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:58:01.214475 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:58:01.214475 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:58:01.214475 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:58:01.214475 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:58:01.214475 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:58:01.214475 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 633/1271] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:58:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:58:01.219030 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aac15a-8efd-496b-a8a2-b2e591b159e9 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:01.279350 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:58:01.320869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aac15a-8efd-496b-a8a2-b2e591b159e9 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:01.354158 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c7eb99-e9c6-44f4-82ec-d432b1db30ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:01.354769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7eb99-e9c6-44f4-82ec-d432b1db30ef None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:01.355080 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7eb99-e9c6-44f4-82ec-d432b1db30ef None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:01.355356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7eb99-e9c6-44f4-82ec-d432b1db30ef None admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:01.356096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c7eb99-e9c6-44f4-82ec-d432b1db30ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:58:01.356404 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c7eb99-e9c6-44f4-82ec-d432b1db30ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:01.357845 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c7eb99-e9c6-44f4-82ec-d432b1db30ef None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:01.369513 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 639/1272] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:58:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:58:01.375581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a81a6-1aa4-4baa-840a-b03610a5014e None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:01.466729 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:58:01.475187 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a81a6-1aa4-4baa-840a-b03610a5014e None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:01.509319 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966bebc8-b9d2-4a6a-bce0-cf108ce11696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:01.509993 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bebc8-b9d2-4a6a-bce0-cf108ce11696 None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:01.510278 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bebc8-b9d2-4a6a-bce0-cf108ce11696 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:01.510558 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bebc8-b9d2-4a6a-bce0-cf108ce11696 None admin] PATH_INFO: `/v3/projects/cf3763a37383456184a9ca9351b4ce03/users/59846d88f4314c2382aa5cb8bae5053b/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:01.512570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966bebc8-b9d2-4a6a-bce0-cf108ce11696 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf3763a37383456184a9ca9351b4ce03, user_id=59846d88f4314c2382aa5cb8bae5053b, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:01.515683 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966bebc8-b9d2-4a6a-bce0-cf108ce11696 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:01.612778 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 634/1273] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Aug 3 04:58:01 2022] PUT /identity/v3/projects/cf3763a37383456184a9ca9351b4ce03/users/59846d88f4314c2382aa5cb8bae5053b/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 04:58:03.605458 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf12e3-c24c-484a-9b11-789ecf0ac679 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:03.606015 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf12e3-c24c-484a-9b11-789ecf0ac679 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:03.606015 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf12e3-c24c-484a-9b11-789ecf0ac679 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:03.607364 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 640/1274] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:58:03.612529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa05beb-d168-407f-a0fa-8c6646fff21c None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:03.612920 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa05beb-d168-407f-a0fa-8c6646fff21c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:03.613094 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa05beb-d168-407f-a0fa-8c6646fff21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:03.630917 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-dfa05beb-d168-407f-a0fa-8c6646fff21c None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:58:03.632138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-dfa05beb-d168-407f-a0fa-8c6646fff21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:58:03.700044 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 635/1275] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:58:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:58:03.723292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf12e3-c24c-484a-9b11-789ecf0ac679 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:03.824914 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf12e3-c24c-484a-9b11-789ecf0ac679 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:03.842706 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:58:03.859763 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e37b35-bbc7-478f-9d7a-a24efdb59cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:03.860401 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e37b35-bbc7-478f-9d7a-a24efdb59cce None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:03.860844 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e37b35-bbc7-478f-9d7a-a24efdb59cce None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:03.861142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e37b35-bbc7-478f-9d7a-a24efdb59cce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:03.871402 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e37b35-bbc7-478f-9d7a-a24efdb59cce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 04:58:03.876296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-47e37b35-bbc7-478f-9d7a-a24efdb59cce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:58:03.876296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:58:03.876296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:58:03.876296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:58:03.876296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:58:03.876296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:58:03.876296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:58:03.876296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:58:03.876296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:58:03.876296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:58:03.876296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:58:03.876296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:58:03.876296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 04:58:03.876296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 04:58:03.876296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 04:58:03.876296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 04:58:03.877442 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 04:58:03.877442 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 04:58:03.877442 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 04:58:03.877442 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 04:58:03.877442 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:58:03.877442 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:58:03.877442 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:58:03.877442 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:58:03.877442 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:58:03.877442 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:58:03.877442 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:58:03.877442 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:58:03.877442 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:58:03.877442 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:58:03.877442 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:58:03.877442 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:58:03.878673 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:58:03.878673 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:58:03.878673 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:58:03.878673 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:58:03.878673 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 04:58:03.878673 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 04:58:03.878673 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 04:58:03.878673 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 04:58:03.878673 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 04:58:03.878673 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 04:58:03.878673 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 04:58:03.878673 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:58:03.878673 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 641/1276] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Aug 3 04:58:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:58:03.884260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa05beb-d168-407f-a0fa-8c6646fff21c None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:03.969004 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:58:03.987504 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa05beb-d168-407f-a0fa-8c6646fff21c None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:04.022469 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067a421-d17e-4d5e-93ec-ee9a6e683a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:04.023104 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067a421-d17e-4d5e-93ec-ee9a6e683a79 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:04.023388 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067a421-d17e-4d5e-93ec-ee9a6e683a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:04.023659 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067a421-d17e-4d5e-93ec-ee9a6e683a79 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:04.024372 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1067a421-d17e-4d5e-93ec-ee9a6e683a79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:58:04.024669 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1067a421-d17e-4d5e-93ec-ee9a6e683a79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:04.026025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1067a421-d17e-4d5e-93ec-ee9a6e683a79 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:04.033887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 636/1277] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Aug 3 04:58:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:58:04.040261 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e37b35-bbc7-478f-9d7a-a24efdb59cce None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:04.142631 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e37b35-bbc7-478f-9d7a-a24efdb59cce None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:04.177622 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94a5ad-3f36-4686-b697-e6b2324a7423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:04.178265 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94a5ad-3f36-4686-b697-e6b2324a7423 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:04.178552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94a5ad-3f36-4686-b697-e6b2324a7423 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:04.178835 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94a5ad-3f36-4686-b697-e6b2324a7423 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:04.189225 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c94a5ad-3f36-4686-b697-e6b2324a7423 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:04.190680 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c94a5ad-3f36-4686-b697-e6b2324a7423 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:04.205404 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-0c94a5ad-3f36-4686-b697-e6b2324a7423 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:58:04.205404 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:58:04.205404 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:58:04.205404 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:58:04.205404 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:58:04.205404 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:58:04.205404 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:58:04.205404 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:58:04.205404 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:58:04.205404 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:58:04.205404 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:58:04.205404 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:58:04.205404 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:58:04.205404 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:58:04.205404 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:58:04.206507 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:58:04.206507 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:58:04.206507 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:58:04.206507 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:58:04.206507 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:58:04.206507 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:58:04.206507 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:58:04.206507 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:58:04.206507 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:58:04.206507 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:58:04.206507 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:58:04.206507 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:58:04.206507 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:58:04.206507 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:58:04.206507 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:58:04.206507 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:58:04.206507 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:58:04.207494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:58:04.207494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:58:04.207494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:58:04.207494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:58:04.207494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 642/1278] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:58:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:58:04.213087 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067a421-d17e-4d5e-93ec-ee9a6e683a79 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:04.312994 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067a421-d17e-4d5e-93ec-ee9a6e683a79 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:04.345992 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24504f3b-3e68-4a0f-8eed-b3cdfece6f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:04.346604 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24504f3b-3e68-4a0f-8eed-b3cdfece6f1a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:04.346976 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24504f3b-3e68-4a0f-8eed-b3cdfece6f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:04.347259 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24504f3b-3e68-4a0f-8eed-b3cdfece6f1a None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:04.347940 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24504f3b-3e68-4a0f-8eed-b3cdfece6f1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:58:04.348244 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24504f3b-3e68-4a0f-8eed-b3cdfece6f1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:04.349596 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24504f3b-3e68-4a0f-8eed-b3cdfece6f1a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:04.361180 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 637/1279] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:58:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:58:04.367076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94a5ad-3f36-4686-b697-e6b2324a7423 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:04.399140 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:58:04.468144 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94a5ad-3f36-4686-b697-e6b2324a7423 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:04.503343 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48be8c-45d6-48fc-8786-7d3b24a6dc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:04.504146 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48be8c-45d6-48fc-8786-7d3b24a6dc26 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:04.504528 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48be8c-45d6-48fc-8786-7d3b24a6dc26 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:04.504937 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48be8c-45d6-48fc-8786-7d3b24a6dc26 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:04.513244 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf48be8c-45d6-48fc-8786-7d3b24a6dc26 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:04.514965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf48be8c-45d6-48fc-8786-7d3b24a6dc26 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:04.526639 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-bf48be8c-45d6-48fc-8786-7d3b24a6dc26 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 03 04:58:04.526639 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:58:04.526639 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:58:04.526639 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:58:04.526639 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:58:04.526639 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:58:04.526639 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:58:04.526639 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:58:04.526639 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:58:04.526639 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:58:04.526639 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:58:04.526639 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:58:04.526639 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 04:58:04.526639 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 04:58:04.526639 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 04:58:04.526639 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 04:58:04.527828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:58:04.527828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:58:04.527828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 04:58:04.527828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:58:04.527828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 04:58:04.527828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 04:58:04.527828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:58:04.527828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:58:04.527828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:58:04.527828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:58:04.527828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:58:04.527828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:58:04.527828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:58:04.527828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:58:04.527828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:58:04.527828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:58:04.527828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:58:04.527828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:58:04.529224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:58:04.529224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:58:04.529224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 04:58:04.529224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 04:58:04.529224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 04:58:04.529224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 04:58:04.529224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 04:58:04.529224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 04:58:04.529224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 03 04:58:04.529224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:58:04.529224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 643/1280] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:58:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:58:04.533685 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24504f3b-3e68-4a0f-8eed-b3cdfece6f1a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:04.537506 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:58:04.631739 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24504f3b-3e68-4a0f-8eed-b3cdfece6f1a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:04.667201 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b5e71-8f77-4c81-82a2-a4c561bf6250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:04.667814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b5e71-8f77-4c81-82a2-a4c561bf6250 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:04.668093 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b5e71-8f77-4c81-82a2-a4c561bf6250 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:04.668363 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b5e71-8f77-4c81-82a2-a4c561bf6250 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:04.669142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692b5e71-8f77-4c81-82a2-a4c561bf6250 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:58:04.669451 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692b5e71-8f77-4c81-82a2-a4c561bf6250 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:04.670884 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692b5e71-8f77-4c81-82a2-a4c561bf6250 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:04.687882 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 638/1281] 10.222.0.40 () {56 vars in 1143 bytes} [Wed Aug 3 04:58:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:58:04.693689 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48be8c-45d6-48fc-8786-7d3b24a6dc26 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:04.784936 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:58:04.794164 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48be8c-45d6-48fc-8786-7d3b24a6dc26 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:04.826697 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00067e4-7009-4202-8aae-2891b7ae6318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:04.827346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00067e4-7009-4202-8aae-2891b7ae6318 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:04.827634 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00067e4-7009-4202-8aae-2891b7ae6318 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:04.827912 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00067e4-7009-4202-8aae-2891b7ae6318 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:04.900274 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00067e4-7009-4202-8aae-2891b7ae6318 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:04.901771 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00067e4-7009-4202-8aae-2891b7ae6318 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:04.991609 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:58:05.002505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 644/1282] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Aug 3 04:58:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 04:58:05.009142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b5e71-8f77-4c81-82a2-a4c561bf6250 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:05.077411 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:58:05.115271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b5e71-8f77-4c81-82a2-a4c561bf6250 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:05.151293 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2ef40b-4e8f-4014-ba33-2b77319dfe06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:05.151928 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ef40b-4e8f-4014-ba33-2b77319dfe06 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:05.152209 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ef40b-4e8f-4014-ba33-2b77319dfe06 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:05.152484 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ef40b-4e8f-4014-ba33-2b77319dfe06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:05.162253 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2ef40b-4e8f-4014-ba33-2b77319dfe06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:05.163726 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2ef40b-4e8f-4014-ba33-2b77319dfe06 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:05.178524 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-8e2ef40b-4e8f-4014-ba33-2b77319dfe06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:58:05.178524 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:58:05.178524 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:58:05.178524 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:58:05.178524 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:58:05.178524 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:58:05.178524 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:58:05.178524 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:58:05.178524 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:58:05.178524 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:58:05.178524 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:58:05.178524 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:58:05.178524 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 04:58:05.178524 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 04:58:05.178524 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 04:58:05.179779 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 04:58:05.179779 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:58:05.179779 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:58:05.179779 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:58:05.179779 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:58:05.179779 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:58:05.179779 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:58:05.179779 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:58:05.179779 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:58:05.179779 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:58:05.179779 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:58:05.179779 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:58:05.179779 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:58:05.179779 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:58:05.179779 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:58:05.179779 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:58:05.179779 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:58:05.181376 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 04:58:05.181376 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 04:58:05.181376 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 04:58:05.181376 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:58:05.181376 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 639/1283] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 04:58:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:58:05.185895 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00067e4-7009-4202-8aae-2891b7ae6318 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:05.295794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00067e4-7009-4202-8aae-2891b7ae6318 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:05.333438 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb89ff9f-4b36-45ab-8e56-0d70e270ffdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:05.334074 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89ff9f-4b36-45ab-8e56-0d70e270ffdd None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:05.334356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89ff9f-4b36-45ab-8e56-0d70e270ffdd None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:05.334630 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89ff9f-4b36-45ab-8e56-0d70e270ffdd None admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:05.335424 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb89ff9f-4b36-45ab-8e56-0d70e270ffdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:58:05.339324 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb89ff9f-4b36-45ab-8e56-0d70e270ffdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:05.339324 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb89ff9f-4b36-45ab-8e56-0d70e270ffdd None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:05.354878 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 645/1284] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Aug 3 04:58:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:58:05.356444 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2ef40b-4e8f-4014-ba33-2b77319dfe06 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:05.467127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2ef40b-4e8f-4014-ba33-2b77319dfe06 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:05.482867 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:58:05.504481 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70965ef6-c138-4d4d-a95c-786b20be98e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:05.505686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70965ef6-c138-4d4d-a95c-786b20be98e1 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:05.505686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70965ef6-c138-4d4d-a95c-786b20be98e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:05.505876 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70965ef6-c138-4d4d-a95c-786b20be98e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:05.513014 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70965ef6-c138-4d4d-a95c-786b20be98e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:05.514472 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70965ef6-c138-4d4d-a95c-786b20be98e1 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:05.527216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-70965ef6-c138-4d4d-a95c-786b20be98e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:58:05.527216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:58:05.527216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:58:05.527216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:58:05.527216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:58:05.527216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:58:05.527216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:58:05.527216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:58:05.527216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:58:05.527216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:58:05.527216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:58:05.527216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:58:05.527216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 04:58:05.527216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 04:58:05.527216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 04:58:05.528270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 04:58:05.528270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:58:05.528270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:58:05.528270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:58:05.528270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:58:05.528270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:58:05.528270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:58:05.528270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:58:05.528270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( Aug 03 04:58:05.528270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:58:05.528270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:58:05.528270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:58:05.528270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:58:05.528270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:58:05.528270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:58:05.528270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:58:05.528270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:58:05.529290 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 04:58:05.529290 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 04:58:05.529290 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 04:58:05.529290 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 04:58:05.529290 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 04:58:05.529290 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 04:58:05.529290 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 04:58:05.529290 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Aug 03 04:58:05.529290 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 640/1285] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Aug 3 04:58:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:58:05.534238 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb89ff9f-4b36-45ab-8e56-0d70e270ffdd None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:05.587987 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:58:05.642975 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb89ff9f-4b36-45ab-8e56-0d70e270ffdd None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:05.683742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b6f4d-4871-4ee2-9011-2f6016f30a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:05.684368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b6f4d-4871-4ee2-9011-2f6016f30a89 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:05.684651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b6f4d-4871-4ee2-9011-2f6016f30a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:05.684929 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b6f4d-4871-4ee2-9011-2f6016f30a89 None admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:05.685679 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b6f4d-4871-4ee2-9011-2f6016f30a89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:58:05.685991 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b6f4d-4871-4ee2-9011-2f6016f30a89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:05.687529 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b6f4d-4871-4ee2-9011-2f6016f30a89 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:05.699674 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 646/1286] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Aug 3 04:58:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:58:05.705765 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70965ef6-c138-4d4d-a95c-786b20be98e1 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:05.806325 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70965ef6-c138-4d4d-a95c-786b20be98e1 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:05.839193 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98238325-4354-469e-aa74-32c0da54e7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:05.839711 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98238325-4354-469e-aa74-32c0da54e7cf None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:05.839993 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98238325-4354-469e-aa74-32c0da54e7cf None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:05.840268 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98238325-4354-469e-aa74-32c0da54e7cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:05.841044 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98238325-4354-469e-aa74-32c0da54e7cf None admin] RBAC: Adding query filter params (role.id=e39d4ecea4c64d5794e1079c9be3ad36, scope.project.id=cf3763a37383456184a9ca9351b4ce03, user.id=59846d88f4314c2382aa5cb8bae5053b) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:58:05.841352 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98238325-4354-469e-aa74-32c0da54e7cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:05.842768 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98238325-4354-469e-aa74-32c0da54e7cf None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:05.848465 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 641/1287] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Aug 3 04:58:05 2022] GET /identity/v3/role_assignments?user.id=59846d88f4314c2382aa5cb8bae5053b&scope.project.id=cf3763a37383456184a9ca9351b4ce03&role.id=e39d4ecea4c64d5794e1079c9be3ad36 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:58:07.923358 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cea388-6c7e-4060-9ccc-cd0c4d151e5f None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:07.923877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cea388-6c7e-4060-9ccc-cd0c4d151e5f None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:07.923877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cea388-6c7e-4060-9ccc-cd0c4d151e5f None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:07.925199 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 647/1288] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:58:07.930971 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a69e0c-aa8e-4665-abea-1ae37296f041 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:07.931400 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a69e0c-aa8e-4665-abea-1ae37296f041 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:07.931530 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a69e0c-aa8e-4665-abea-1ae37296f041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:07.949887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-42a69e0c-aa8e-4665-abea-1ae37296f041 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:58:07.951115 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-42a69e0c-aa8e-4665-abea-1ae37296f041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:58:07.980484 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:58:08.017622 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 642/1289] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:58:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 04:58:08.043691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cea388-6c7e-4060-9ccc-cd0c4d151e5f None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:08.158041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cea388-6c7e-4060-9ccc-cd0c4d151e5f None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:08.168592 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:58:08.191624 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13ff75-ff21-4981-89f7-aa4cad188a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:08.192210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13ff75-ff21-4981-89f7-aa4cad188a74 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:08.192577 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13ff75-ff21-4981-89f7-aa4cad188a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:08.192859 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13ff75-ff21-4981-89f7-aa4cad188a74 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:08.193484 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c13ff75-ff21-4981-89f7-aa4cad188a74 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:08.194909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c13ff75-ff21-4981-89f7-aa4cad188a74 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:08.204408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-3c13ff75-ff21-4981-89f7-aa4cad188a74 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 04:58:08.204408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:58:08.204408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 04:58:08.204408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 04:58:08.204408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 04:58:08.204408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 04:58:08.204408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 04:58:08.204408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:58:08.204408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 04:58:08.204408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:58:08.204408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:58:08.204408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:58:08.204408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:58:08.204408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:58:08.204408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:58:08.204408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:58:08.204408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:58:08.205654 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:58:08.205654 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:58:08.205654 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:58:08.205654 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:58:08.205654 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:58:08.205654 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:58:08.205654 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 04:58:08.205654 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 04:58:08.205654 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 04:58:08.205654 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 04:58:08.205654 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:58:08.205654 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:58:08.205654 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:58:08.205654 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:58:08.205654 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:58:08.205654 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:58:08.206734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:58:08.206734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:58:08.206734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:58:08.206734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:58:08.206734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:58:08.206734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:58:08.206734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:58:08.206734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:58:08.206734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:58:08.206734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:58:08.206734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 04:58:08.206734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:58:08.206734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 04:58:08.206734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:58:08.206734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 648/1290] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Aug 3 04:58:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:58:08.211623 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a69e0c-aa8e-4665-abea-1ae37296f041 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:08.308756 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a69e0c-aa8e-4665-abea-1ae37296f041 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:08.340945 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c3fc5-3829-4366-acb8-8d14d5278ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:08.341553 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c3fc5-3829-4366-acb8-8d14d5278ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:08.341833 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c3fc5-3829-4366-acb8-8d14d5278ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:08.342104 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c3fc5-3829-4366-acb8-8d14d5278ae9 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:08.342770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c3fc5-3829-4366-acb8-8d14d5278ae9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:58:08.343103 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c3fc5-3829-4366-acb8-8d14d5278ae9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:08.344454 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c3fc5-3829-4366-acb8-8d14d5278ae9 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:08.349204 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 643/1291] 10.222.0.40 () {56 vars in 1116 bytes} [Wed Aug 3 04:58:08 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:58:08.355735 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13ff75-ff21-4981-89f7-aa4cad188a74 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:08.456273 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13ff75-ff21-4981-89f7-aa4cad188a74 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:08.489386 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60bd78-1997-41ed-90fe-29e3b15642c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:08.489988 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60bd78-1997-41ed-90fe-29e3b15642c8 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:08.490265 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60bd78-1997-41ed-90fe-29e3b15642c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:08.490580 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60bd78-1997-41ed-90fe-29e3b15642c8 None admin] PATH_INFO: `/v3/services` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:08.491271 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60bd78-1997-41ed-90fe-29e3b15642c8 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:58:08.495607 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60bd78-1997-41ed-90fe-29e3b15642c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:08.495607 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60bd78-1997-41ed-90fe-29e3b15642c8 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:08.497999 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 649/1292] 10.222.0.40 () {56 vars in 1116 bytes} [Wed Aug 3 04:58:08 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:58:10.526775 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca097daf-339c-49bf-b90f-6ffff9262a02 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:10.527452 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca097daf-339c-49bf-b90f-6ffff9262a02 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:10.527452 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca097daf-339c-49bf-b90f-6ffff9262a02 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:10.528704 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 644/1293] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:58:10.534066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2bb160-76c1-4db3-82da-2198291c4179 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:10.557215 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2bb160-76c1-4db3-82da-2198291c4179 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:10.557215 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2bb160-76c1-4db3-82da-2198291c4179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:10.557215 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-1f2bb160-76c1-4db3-82da-2198291c4179 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:58:10.557215 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-1f2bb160-76c1-4db3-82da-2198291c4179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:58:10.583280 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:58:10.619474 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 650/1294] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:58:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:58:10.642434 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca097daf-339c-49bf-b90f-6ffff9262a02 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:10.674864 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:58:10.744780 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca097daf-339c-49bf-b90f-6ffff9262a02 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:10.778734 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e2f28-0c61-4185-ab03-51b4e997a4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:10.779301 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e2f28-0c61-4185-ab03-51b4e997a4d0 None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:10.779599 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e2f28-0c61-4185-ab03-51b4e997a4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:10.779873 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e2f28-0c61-4185-ab03-51b4e997a4d0 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:10.780562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e2f28-0c61-4185-ab03-51b4e997a4d0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:10.781915 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e2f28-0c61-4185-ab03-51b4e997a4d0 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:10.897603 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 645/1295] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Aug 3 04:58:10 2022] POST /identity/v3/services => generated 246 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:58:12.869085 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1510abb-2bdf-4bbf-82a2-49d64a42af4e None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:12.869645 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1510abb-2bdf-4bbf-82a2-49d64a42af4e None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:12.869645 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1510abb-2bdf-4bbf-82a2-49d64a42af4e None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:12.871044 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 651/1296] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:58:12.876303 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae9159-edab-4384-b573-cca9aad31ab4 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:12.876699 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae9159-edab-4384-b573-cca9aad31ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:12.876870 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae9159-edab-4384-b573-cca9aad31ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:12.895838 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-15ae9159-edab-4384-b573-cca9aad31ab4 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:58:12.897038 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-15ae9159-edab-4384-b573-cca9aad31ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:58:12.969531 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 646/1297] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:58:12 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:58:12.992081 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1510abb-2bdf-4bbf-82a2-49d64a42af4e None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:13.094604 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1510abb-2bdf-4bbf-82a2-49d64a42af4e None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:13.128359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f6da0-35aa-4149-aa1a-0afad53a6863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:13.128989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f6da0-35aa-4149-aa1a-0afad53a6863 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:13.129274 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f6da0-35aa-4149-aa1a-0afad53a6863 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:13.129551 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f6da0-35aa-4149-aa1a-0afad53a6863 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:13.130175 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f6da0-35aa-4149-aa1a-0afad53a6863 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:13.131746 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f6da0-35aa-4149-aa1a-0afad53a6863 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:13.141215 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-ca9f6da0-35aa-4149-aa1a-0afad53a6863 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 04:58:13.141215 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:58:13.141215 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 04:58:13.141215 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 04:58:13.141215 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 04:58:13.141215 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 04:58:13.141215 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 04:58:13.141215 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:58:13.141215 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 04:58:13.141215 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:58:13.141215 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:58:13.141215 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:58:13.141215 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:58:13.141215 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:58:13.141215 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:58:13.141215 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:58:13.141215 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:58:13.142374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:58:13.142374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:58:13.142374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:58:13.142374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:58:13.142374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:58:13.142374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:58:13.142374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 04:58:13.142374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 04:58:13.142374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 04:58:13.142374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 04:58:13.142374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:58:13.142374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:58:13.142374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:58:13.142374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:58:13.142374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:58:13.142374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:58:13.143662 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:58:13.143662 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:58:13.143662 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:58:13.143662 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:58:13.143662 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:58:13.143662 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:58:13.143662 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:58:13.143662 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:58:13.143662 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:58:13.143662 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:58:13.143662 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 04:58:13.143662 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:58:13.143662 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 04:58:13.143662 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:58:13.143662 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 652/1298] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Aug 3 04:58:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:58:13.148397 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae9159-edab-4384-b573-cca9aad31ab4 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:13.166752 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:58:13.247198 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae9159-edab-4384-b573-cca9aad31ab4 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:13.287358 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab43a81-ddca-4f4f-be90-dbd0fff5483a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:13.287982 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab43a81-ddca-4f4f-be90-dbd0fff5483a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:13.288264 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab43a81-ddca-4f4f-be90-dbd0fff5483a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:13.288542 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab43a81-ddca-4f4f-be90-dbd0fff5483a None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:13.289237 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab43a81-ddca-4f4f-be90-dbd0fff5483a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:58:13.289531 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab43a81-ddca-4f4f-be90-dbd0fff5483a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:13.290964 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab43a81-ddca-4f4f-be90-dbd0fff5483a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:13.296267 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 647/1299] 10.222.0.40 () {56 vars in 1116 bytes} [Wed Aug 3 04:58:13 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:58:13.302344 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f6da0-35aa-4149-aa1a-0afad53a6863 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:13.334201 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:58:13.400998 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f6da0-35aa-4149-aa1a-0afad53a6863 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:13.434783 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04b821-f5c5-468c-ac13-47672f1ea02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:13.435573 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04b821-f5c5-468c-ac13-47672f1ea02c None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:13.435860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04b821-f5c5-468c-ac13-47672f1ea02c None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:13.436135 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04b821-f5c5-468c-ac13-47672f1ea02c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:13.436914 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04b821-f5c5-468c-ac13-47672f1ea02c None admin] RBAC: Adding query filter params (interface=public, service_id=c9fc85839a9945989cd64dfd980018c7, region_id=RegionOne) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:58:13.437226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04b821-f5c5-468c-ac13-47672f1ea02c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:13.438596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04b821-f5c5-468c-ac13-47672f1ea02c None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:13.443677 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 653/1300] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Aug 3 04:58:13 2022] GET /identity/v3/endpoints?service_id=c9fc85839a9945989cd64dfd980018c7&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:58:13.450142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab43a81-ddca-4f4f-be90-dbd0fff5483a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:13.531720 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:58:13.549408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab43a81-ddca-4f4f-be90-dbd0fff5483a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:13.583570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d5b07e-35cc-4047-9917-e6e570d7be2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:13.584203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5b07e-35cc-4047-9917-e6e570d7be2a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:13.584488 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5b07e-35cc-4047-9917-e6e570d7be2a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:13.584765 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5b07e-35cc-4047-9917-e6e570d7be2a None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:13.585417 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d5b07e-35cc-4047-9917-e6e570d7be2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:13.587198 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d5b07e-35cc-4047-9917-e6e570d7be2a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:13.596326 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 648/1301] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Aug 3 04:58:13 2022] GET /identity/v3/services => generated 2191 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 04:58:15.652002 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab8fea-f352-4b3e-aa1f-9cc863d552bf None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:15.652571 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab8fea-f352-4b3e-aa1f-9cc863d552bf None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:15.652571 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab8fea-f352-4b3e-aa1f-9cc863d552bf None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:15.653896 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 654/1302] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 04:58:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 04:58:15.659299 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b782980-3413-4703-bf06-503d887072ac None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:15.659735 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b782980-3413-4703-bf06-503d887072ac None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:15.659870 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b782980-3413-4703-bf06-503d887072ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:15.678308 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-7b782980-3413-4703-bf06-503d887072ac None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 04:58:15.686089 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7b782980-3413-4703-bf06-503d887072ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 04:58:15.744289 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 649/1303] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 04:58:15 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 04:58:15.767181 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab8fea-f352-4b3e-aa1f-9cc863d552bf None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:15.864939 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:58:15.871749 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab8fea-f352-4b3e-aa1f-9cc863d552bf None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:15.905444 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b5ddc4-92a5-44c5-83f6-0e0bf157da84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:15.906046 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5ddc4-92a5-44c5-83f6-0e0bf157da84 None admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:15.906333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5ddc4-92a5-44c5-83f6-0e0bf157da84 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:15.906612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5ddc4-92a5-44c5-83f6-0e0bf157da84 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:15.907272 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b5ddc4-92a5-44c5-83f6-0e0bf157da84 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:15.908657 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b5ddc4-92a5-44c5-83f6-0e0bf157da84 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:15.917505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application [None req-79b5ddc4-92a5-44c5-83f6-0e0bf157da84 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 04:58:15.917505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:58:15.917505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 04:58:15.917505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 04:58:15.917505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 04:58:15.917505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 04:58:15.917505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 04:58:15.917505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:58:15.917505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 04:58:15.917505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:58:15.917505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 04:58:15.917505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:58:15.917505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 04:58:15.917505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 04:58:15.917505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 04:58:15.917505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 04:58:15.917505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 04:58:15.922999 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 04:58:15.922999 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 04:58:15.922999 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 04:58:15.922999 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 04:58:15.922999 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 04:58:15.922999 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 04:58:15.922999 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 04:58:15.922999 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 04:58:15.922999 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 04:58:15.922999 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 04:58:15.922999 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 04:58:15.922999 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 04:58:15.922999 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 04:58:15.922999 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 04:58:15.922999 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 04:58:15.922999 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 04:58:15.924112 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 04:58:15.924112 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application with Lock( Aug 03 04:58:15.924112 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 04:58:15.924112 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self._enter() Aug 03 04:58:15.924112 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 04:58:15.924112 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 04:58:15.924112 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 04:58:15.924112 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application return self.creator() Aug 03 04:58:15.924112 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 04:58:15.924112 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application created_value = creator( Aug 03 04:58:15.924112 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 04:58:15.924112 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 04:58:15.924112 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 04:58:15.924112 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: ERROR keystone.server.flask.application Aug 03 04:58:15.924112 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 655/1304] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Aug 3 04:58:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 04:58:15.924949 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b782980-3413-4703-bf06-503d887072ac None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:16.000628 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:58:16.033470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b782980-3413-4703-bf06-503d887072ac None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:16.067956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebcaf32-6c84-486b-afdd-fac19a5269ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:16.068695 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebcaf32-6c84-486b-afdd-fac19a5269ec None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:16.069035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebcaf32-6c84-486b-afdd-fac19a5269ec None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:16.069377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebcaf32-6c84-486b-afdd-fac19a5269ec None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:16.070050 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebcaf32-6c84-486b-afdd-fac19a5269ec None admin] RBAC: Adding query filter params (name=placement) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 04:58:16.070353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebcaf32-6c84-486b-afdd-fac19a5269ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:16.072462 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebcaf32-6c84-486b-afdd-fac19a5269ec None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:16.077668 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 650/1305] 10.222.0.40 () {56 vars in 1116 bytes} [Wed Aug 3 04:58:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 04:58:16.083644 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b5ddc4-92a5-44c5-83f6-0e0bf157da84 None admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 04:58:16.188440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b5ddc4-92a5-44c5-83f6-0e0bf157da84 None admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 04:58:16.225009 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a226ec-662a-414f-aaab-b5b36fd98161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 04:58:16.225627 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a226ec-662a-414f-aaab-b5b36fd98161 None admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 04:58:16.225915 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a226ec-662a-414f-aaab-b5b36fd98161 None admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 04:58:16.226195 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a226ec-662a-414f-aaab-b5b36fd98161 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 04:58:16.226936 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a226ec-662a-414f-aaab-b5b36fd98161 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 04:58:16.228355 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a226ec-662a-414f-aaab-b5b36fd98161 None admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 04:58:16.309763 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 656/1306] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 04:58:16 2022] POST /identity/v3/endpoints => generated 331 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 04:58:26.074592 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=30 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 04:58:26.304550 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:04:20.170946 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b469dce-2201-4118-b679-b69ef2d9620e None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:20.171523 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b469dce-2201-4118-b679-b69ef2d9620e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:20.171523 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b469dce-2201-4118-b679-b69ef2d9620e None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:20.175406 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 651/1307] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:04:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:04:20.178111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15544c2-497b-4775-b003-4da10c40cc05 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:20.178588 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15544c2-497b-4775-b003-4da10c40cc05 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:20.178676 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15544c2-497b-4775-b003-4da10c40cc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:20.266922 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-d15544c2-497b-4775-b003-4da10c40cc05 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:04:20.268197 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-d15544c2-497b-4775-b003-4da10c40cc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:20.399314 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 657/1308] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:04:20.437144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661203bc-654e-4cbd-b3bd-0e0480875baa None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:20.437650 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661203bc-654e-4cbd-b3bd-0e0480875baa None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:20.438111 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661203bc-654e-4cbd-b3bd-0e0480875baa None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:20.440013 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 652/1309] 10.222.0.40 () {54 vars in 965 bytes} [Wed Aug 3 05:04:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:04:20.447919 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c61836-d8bf-4f99-a4c5-67bc1f872aec None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:20.448469 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c61836-d8bf-4f99-a4c5-67bc1f872aec None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:20.448962 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c61836-d8bf-4f99-a4c5-67bc1f872aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:20.597165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-52c61836-d8bf-4f99-a4c5-67bc1f872aec None None] MFA Rules not processed for user `cff15ea4cbc64b068aa5f6586b5bff87`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:04:20.598709 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-52c61836-d8bf-4f99-a4c5-67bc1f872aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:20.623724 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:04:20.691148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 658/1310] 10.222.0.40 () {58 vars in 1062 bytes} [Wed Aug 3 05:04:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:04:20.709212 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661203bc-654e-4cbd-b3bd-0e0480875baa None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:04:20.711308 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-661203bc-654e-4cbd-b3bd-0e0480875baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:20.841132 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661203bc-654e-4cbd-b3bd-0e0480875baa None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:04:20.930554 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125badbc-82e0-4eaf-acf7-ca11ce309a6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:04:20.931126 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125badbc-82e0-4eaf-acf7-ca11ce309a6c service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:20.931406 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125badbc-82e0-4eaf-acf7-ca11ce309a6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:20.931679 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125badbc-82e0-4eaf-acf7-ca11ce309a6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:20.933459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-125badbc-82e0-4eaf-acf7-ca11ce309a6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:20.992865 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:04:21.050870 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125badbc-82e0-4eaf-acf7-ca11ce309a6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:04:21.052426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125badbc-82e0-4eaf-acf7-ca11ce309a6c service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:04:21.071565 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 653/1311] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:04:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:04:23.416062 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5c509-ea52-4be6-ac73-a018e8f45bc4 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:23.416599 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5c509-ea52-4be6-ac73-a018e8f45bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:23.416713 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5c509-ea52-4be6-ac73-a018e8f45bc4 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:23.418014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 659/1312] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:04:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:04:23.423376 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037c7b0-8a2f-42fa-878d-7d4bb74ff746 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:23.423825 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037c7b0-8a2f-42fa-878d-7d4bb74ff746 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:23.423941 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037c7b0-8a2f-42fa-878d-7d4bb74ff746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:23.515433 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-a037c7b0-8a2f-42fa-878d-7d4bb74ff746 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:04:23.516835 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a037c7b0-8a2f-42fa-878d-7d4bb74ff746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:23.618306 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 654/1313] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:04:23.657783 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5c509-ea52-4be6-ac73-a018e8f45bc4 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:04:23.659392 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-afe5c509-ea52-4be6-ac73-a018e8f45bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:23.768839 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5c509-ea52-4be6-ac73-a018e8f45bc4 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:04:23.844928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:04:23.847554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5124b17e-d9d4-4f42-88e3-82ba8dea9258 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:04:23.848169 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124b17e-d9d4-4f42-88e3-82ba8dea9258 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:23.848519 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124b17e-d9d4-4f42-88e3-82ba8dea9258 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:23.848709 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124b17e-d9d4-4f42-88e3-82ba8dea9258 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:23.850350 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-5124b17e-d9d4-4f42-88e3-82ba8dea9258 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:23.959633 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5124b17e-d9d4-4f42-88e3-82ba8dea9258 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:04:23.961079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5124b17e-d9d4-4f42-88e3-82ba8dea9258 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:04:23.978720 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 660/1314] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:04:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:04:26.476813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d55ab3-8abe-4301-bf24-c72aa17f7c0c None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:26.477362 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d55ab3-8abe-4301-bf24-c72aa17f7c0c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:26.477362 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d55ab3-8abe-4301-bf24-c72aa17f7c0c None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:26.478732 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 655/1315] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:04:26.484538 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748925ef-5898-443e-a6e8-3048d15ec129 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:26.484958 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748925ef-5898-443e-a6e8-3048d15ec129 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:26.485120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748925ef-5898-443e-a6e8-3048d15ec129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:26.552646 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-748925ef-5898-443e-a6e8-3048d15ec129 None None] MFA Rules not processed for user `b2cf0210955141068b832fc376bdc5ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:04:26.553866 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-748925ef-5898-443e-a6e8-3048d15ec129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:26.604851 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:04:26.649663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 661/1316] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:04:26 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:04:26.689624 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d55ab3-8abe-4301-bf24-c72aa17f7c0c None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:04:26.691176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e1d55ab3-8abe-4301-bf24-c72aa17f7c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:26.700908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:04:26.798532 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d55ab3-8abe-4301-bf24-c72aa17f7c0c None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:04:26.870860 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2032783-9a24-418a-94e8-54fd278a6d81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:04:26.871518 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2032783-9a24-418a-94e8-54fd278a6d81 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:26.871820 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2032783-9a24-418a-94e8-54fd278a6d81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:26.872097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2032783-9a24-418a-94e8-54fd278a6d81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:26.873963 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e2032783-9a24-418a-94e8-54fd278a6d81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:26.941763 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:04:26.992473 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2032783-9a24-418a-94e8-54fd278a6d81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:04:26.993915 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2032783-9a24-418a-94e8-54fd278a6d81 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:04:27.012703 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 656/1317] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:04:26 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:04:29.805920 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f661dc-7115-443b-b8cb-def8d0131859 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:29.806446 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f661dc-7115-443b-b8cb-def8d0131859 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:29.806568 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f661dc-7115-443b-b8cb-def8d0131859 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:29.807921 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 662/1318] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:04:29.813013 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02799d-b8f1-4d48-80aa-0f34d035b65f None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:29.813418 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02799d-b8f1-4d48-80aa-0f34d035b65f None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:29.813571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02799d-b8f1-4d48-80aa-0f34d035b65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:29.880206 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-ca02799d-b8f1-4d48-80aa-0f34d035b65f None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:04:29.881479 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ca02799d-b8f1-4d48-80aa-0f34d035b65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:29.977187 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 657/1319] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:04:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:04:30.016480 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f661dc-7115-443b-b8cb-def8d0131859 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:04:30.018052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-17f661dc-7115-443b-b8cb-def8d0131859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:30.118573 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f661dc-7115-443b-b8cb-def8d0131859 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:04:30.190811 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a5d79-2b33-4d5d-8fb8-bb9f42c7d299 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:04:30.191450 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a5d79-2b33-4d5d-8fb8-bb9f42c7d299 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:30.191732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a5d79-2b33-4d5d-8fb8-bb9f42c7d299 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:30.192003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a5d79-2b33-4d5d-8fb8-bb9f42c7d299 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:30.193623 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-e64a5d79-2b33-4d5d-8fb8-bb9f42c7d299 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:30.201416 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:04:30.303999 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64a5d79-2b33-4d5d-8fb8-bb9f42c7d299 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:04:30.305461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64a5d79-2b33-4d5d-8fb8-bb9f42c7d299 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:04:30.324251 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 663/1320] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:04:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:04:33.355534 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccaa4ac-3066-4d61-b1e3-1cebb871fd30 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:33.356078 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccaa4ac-3066-4d61-b1e3-1cebb871fd30 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:33.356078 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccaa4ac-3066-4d61-b1e3-1cebb871fd30 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:33.357462 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 658/1321] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:04:33.362664 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f7d62-88ac-46b7-8e96-54f4cd4b7213 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:33.363087 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f7d62-88ac-46b7-8e96-54f4cd4b7213 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:33.363263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f7d62-88ac-46b7-8e96-54f4cd4b7213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:33.429132 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-721f7d62-88ac-46b7-8e96-54f4cd4b7213 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:04:33.430374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-721f7d62-88ac-46b7-8e96-54f4cd4b7213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:33.488468 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:04:33.528425 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 664/1322] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:04:33.578992 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b311ad-7099-4bf2-9bfa-e5838a812814 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:33.579343 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b311ad-7099-4bf2-9bfa-e5838a812814 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:33.579684 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b311ad-7099-4bf2-9bfa-e5838a812814 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:33.581286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 659/1323] 10.222.0.40 () {54 vars in 965 bytes} [Wed Aug 3 05:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:04:33.587756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021724c8-c60b-4a1e-95b0-c66d59aeaf20 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:33.588147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021724c8-c60b-4a1e-95b0-c66d59aeaf20 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:33.588429 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021724c8-c60b-4a1e-95b0-c66d59aeaf20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:33.672283 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-021724c8-c60b-4a1e-95b0-c66d59aeaf20 None None] MFA Rules not processed for user `cff15ea4cbc64b068aa5f6586b5bff87`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:04:33.673535 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-021724c8-c60b-4a1e-95b0-c66d59aeaf20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:33.754486 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 665/1324] 10.222.0.40 () {58 vars in 1062 bytes} [Wed Aug 3 05:04:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:04:33.763578 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b311ad-7099-4bf2-9bfa-e5838a812814 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:04:33.765182 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-07b311ad-7099-4bf2-9bfa-e5838a812814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:33.784772 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:04:33.871050 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b311ad-7099-4bf2-9bfa-e5838a812814 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:04:33.945103 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06affa75-2e0f-473b-9804-4a3e8cc992b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:04:33.945614 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06affa75-2e0f-473b-9804-4a3e8cc992b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:33.945899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06affa75-2e0f-473b-9804-4a3e8cc992b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:33.946175 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06affa75-2e0f-473b-9804-4a3e8cc992b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:33.947858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-06affa75-2e0f-473b-9804-4a3e8cc992b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:34.020600 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:04:34.059700 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06affa75-2e0f-473b-9804-4a3e8cc992b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:04:34.061148 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06affa75-2e0f-473b-9804-4a3e8cc992b5 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:04:34.081202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 660/1325] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:04:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:04:36.362637 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aecbd40-fcd6-4138-92e0-e958b07c92d5 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:36.363141 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aecbd40-fcd6-4138-92e0-e958b07c92d5 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:36.363235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aecbd40-fcd6-4138-92e0-e958b07c92d5 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:36.364592 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 666/1326] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:04:36.370139 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f1a93-9149-4c80-bd23-61492ab36977 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:36.370665 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f1a93-9149-4c80-bd23-61492ab36977 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:36.370665 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f1a93-9149-4c80-bd23-61492ab36977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:36.440324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-676f1a93-9149-4c80-bd23-61492ab36977 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:04:36.441594 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-676f1a93-9149-4c80-bd23-61492ab36977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:36.543774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 661/1327] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:04:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:04:36.579241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aecbd40-fcd6-4138-92e0-e958b07c92d5 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:04:36.580780 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-2aecbd40-fcd6-4138-92e0-e958b07c92d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:36.653928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:04:36.698180 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aecbd40-fcd6-4138-92e0-e958b07c92d5 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:04:36.773645 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c5545-cd9a-4bae-a1e7-85d9f6d36c94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:04:36.774263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c5545-cd9a-4bae-a1e7-85d9f6d36c94 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:36.774539 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c5545-cd9a-4bae-a1e7-85d9f6d36c94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:36.774810 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c5545-cd9a-4bae-a1e7-85d9f6d36c94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:36.777079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-847c5545-cd9a-4bae-a1e7-85d9f6d36c94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:36.895761 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:04:36.898220 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847c5545-cd9a-4bae-a1e7-85d9f6d36c94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:04:36.899715 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847c5545-cd9a-4bae-a1e7-85d9f6d36c94 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:04:36.918762 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 667/1328] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:04:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:04:38.987412 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a4107-8922-4dd2-90c4-7a059ac5a560 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:38.988004 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a4107-8922-4dd2-90c4-7a059ac5a560 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:38.988004 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a4107-8922-4dd2-90c4-7a059ac5a560 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:38.989337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 662/1329] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:04:38.995092 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c070c0-f96a-46ca-b002-f721c56be297 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:38.995552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c070c0-f96a-46ca-b002-f721c56be297 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:38.995808 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c070c0-f96a-46ca-b002-f721c56be297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:39.073967 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-73c070c0-f96a-46ca-b002-f721c56be297 None None] MFA Rules not processed for user `b2cf0210955141068b832fc376bdc5ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:04:39.075127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-73c070c0-f96a-46ca-b002-f721c56be297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:39.173597 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 668/1330] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:04:38 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:04:39.210416 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a4107-8922-4dd2-90c4-7a059ac5a560 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:04:39.212072 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d15a4107-8922-4dd2-90c4-7a059ac5a560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:39.243385 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:04:39.331811 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a4107-8922-4dd2-90c4-7a059ac5a560 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:04:39.413691 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d304382-af51-4198-b27e-6d82fcd66ecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:04:39.414314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d304382-af51-4198-b27e-6d82fcd66ecd service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:39.414592 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d304382-af51-4198-b27e-6d82fcd66ecd service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:39.414900 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d304382-af51-4198-b27e-6d82fcd66ecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:39.417124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8d304382-af51-4198-b27e-6d82fcd66ecd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:39.494167 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:04:39.525870 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d304382-af51-4198-b27e-6d82fcd66ecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:04:39.527390 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d304382-af51-4198-b27e-6d82fcd66ecd service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:04:39.547435 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 663/1331] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:04:39 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:04:42.362757 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0839b5e-fadb-4068-ba7b-c5987b27224a None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:42.363761 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0839b5e-fadb-4068-ba7b-c5987b27224a None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:42.363761 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0839b5e-fadb-4068-ba7b-c5987b27224a None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:42.365586 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 669/1332] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:04:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:04:42.373021 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09c1bf-3d1e-488b-8ab7-e28dd9eea18b None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:42.373486 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09c1bf-3d1e-488b-8ab7-e28dd9eea18b None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:42.373623 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09c1bf-3d1e-488b-8ab7-e28dd9eea18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:42.443259 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-6e09c1bf-3d1e-488b-8ab7-e28dd9eea18b None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:04:42.444471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6e09c1bf-3d1e-488b-8ab7-e28dd9eea18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:42.539909 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 664/1333] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:04:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:04:42.577546 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0839b5e-fadb-4068-ba7b-c5987b27224a None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:04:42.579098 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-f0839b5e-fadb-4068-ba7b-c5987b27224a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:42.648789 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:04:42.684290 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0839b5e-fadb-4068-ba7b-c5987b27224a None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:04:42.758487 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5647d6b-f0ac-4c18-b635-b83cfc8c7661 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:04:42.759134 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5647d6b-f0ac-4c18-b635-b83cfc8c7661 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:42.759418 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5647d6b-f0ac-4c18-b635-b83cfc8c7661 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:42.759692 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5647d6b-f0ac-4c18-b635-b83cfc8c7661 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:42.761321 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-f5647d6b-f0ac-4c18-b635-b83cfc8c7661 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:42.868496 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5647d6b-f0ac-4c18-b635-b83cfc8c7661 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:04:42.870584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5647d6b-f0ac-4c18-b635-b83cfc8c7661 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:04:42.880799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:04:42.889430 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 670/1334] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:04:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:04:44.935878 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc05b65-ac3d-4ad2-9a7a-4f0764ba9039 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:44.936489 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc05b65-ac3d-4ad2-9a7a-4f0764ba9039 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:44.936489 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc05b65-ac3d-4ad2-9a7a-4f0764ba9039 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:44.937728 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 665/1335] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:04:44.945903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91debd60-f634-4089-9bca-8e1a13f55420 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:44.946483 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91debd60-f634-4089-9bca-8e1a13f55420 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:44.946483 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91debd60-f634-4089-9bca-8e1a13f55420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:45.012496 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-91debd60-f634-4089-9bca-8e1a13f55420 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:04:45.013750 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-91debd60-f634-4089-9bca-8e1a13f55420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:45.114294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 671/1336] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:04:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:04:45.154893 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc05b65-ac3d-4ad2-9a7a-4f0764ba9039 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:04:45.156435 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6bc05b65-ac3d-4ad2-9a7a-4f0764ba9039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:45.196044 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:04:45.260648 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc05b65-ac3d-4ad2-9a7a-4f0764ba9039 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:04:45.333223 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52700e5-0f13-454e-bd87-8eed02204f4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:04:45.333834 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52700e5-0f13-454e-bd87-8eed02204f4a service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:45.334115 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52700e5-0f13-454e-bd87-8eed02204f4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:45.334387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52700e5-0f13-454e-bd87-8eed02204f4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:45.336060 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d52700e5-0f13-454e-bd87-8eed02204f4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:45.418296 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:04:45.443566 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52700e5-0f13-454e-bd87-8eed02204f4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:04:45.444999 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52700e5-0f13-454e-bd87-8eed02204f4a service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:04:45.463714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 666/1337] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:04:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:04:48.230536 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6fdc7-e384-45a8-a49d-59dd6932292b None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:48.231150 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6fdc7-e384-45a8-a49d-59dd6932292b None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:48.231150 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6fdc7-e384-45a8-a49d-59dd6932292b None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:48.232497 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 672/1338] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:04:48.238244 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650b596-c2e9-45da-b96d-8d56f98e28ea None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:48.238704 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650b596-c2e9-45da-b96d-8d56f98e28ea None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:48.238798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650b596-c2e9-45da-b96d-8d56f98e28ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:48.305498 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-c650b596-c2e9-45da-b96d-8d56f98e28ea None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:04:48.307286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c650b596-c2e9-45da-b96d-8d56f98e28ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:48.403267 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 667/1339] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:04:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:04:48.440035 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c6fdc7-e384-45a8-a49d-59dd6932292b None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:04:48.441563 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-40c6fdc7-e384-45a8-a49d-59dd6932292b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:48.517063 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:04:48.543842 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c6fdc7-e384-45a8-a49d-59dd6932292b None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:04:48.616820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7e09d-1dd1-4d6d-8454-fcc16dca9764 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:04:48.617435 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7e09d-1dd1-4d6d-8454-fcc16dca9764 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:48.617717 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7e09d-1dd1-4d6d-8454-fcc16dca9764 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:48.617994 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7e09d-1dd1-4d6d-8454-fcc16dca9764 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:48.620210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-b2d7e09d-1dd1-4d6d-8454-fcc16dca9764 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:48.728619 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d7e09d-1dd1-4d6d-8454-fcc16dca9764 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:04:48.730068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d7e09d-1dd1-4d6d-8454-fcc16dca9764 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:04:48.751129 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 673/1340] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:04:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:04:53.058571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f2aa4-5da5-4105-875c-40cf9d7a0b35 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:53.059064 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f2aa4-5da5-4105-875c-40cf9d7a0b35 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:53.059233 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f2aa4-5da5-4105-875c-40cf9d7a0b35 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:53.061245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 668/1341] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:04:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:04:53.066506 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7c2b9-60ca-410f-b98e-7f0bb6dc33fd None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:53.066888 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7c2b9-60ca-410f-b98e-7f0bb6dc33fd None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:53.067055 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7c2b9-60ca-410f-b98e-7f0bb6dc33fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:53.078128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:04:53.141330 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-3cf7c2b9-60ca-410f-b98e-7f0bb6dc33fd None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:04:53.142599 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-3cf7c2b9-60ca-410f-b98e-7f0bb6dc33fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:53.240557 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 674/1342] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:04:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:04:53.279751 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f2aa4-5da5-4105-875c-40cf9d7a0b35 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:04:53.281294 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e32f2aa4-5da5-4105-875c-40cf9d7a0b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:53.330162 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:04:53.386349 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f2aa4-5da5-4105-875c-40cf9d7a0b35 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:04:53.458306 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9ac3c-6097-4e1a-9860-d55812858d9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:04:53.458960 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9ac3c-6097-4e1a-9860-d55812858d9f service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:53.459245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9ac3c-6097-4e1a-9860-d55812858d9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:53.459522 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9ac3c-6097-4e1a-9860-d55812858d9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:53.461137 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-76a9ac3c-6097-4e1a-9860-d55812858d9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:53.555956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:04:53.572290 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a9ac3c-6097-4e1a-9860-d55812858d9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:04:53.573755 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a9ac3c-6097-4e1a-9860-d55812858d9f service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:04:53.593135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 669/1343] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:04:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:04:56.576516 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff70b88-7c22-493f-989a-df9ae5dbb4b2 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:56.577046 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff70b88-7c22-493f-989a-df9ae5dbb4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:56.577161 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff70b88-7c22-493f-989a-df9ae5dbb4b2 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:56.578429 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 675/1344] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:04:56.583756 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb3141-c116-4eaf-86a0-81d1938bb0bf None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:56.584207 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb3141-c116-4eaf-86a0-81d1938bb0bf None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:56.584321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb3141-c116-4eaf-86a0-81d1938bb0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:56.651393 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-37fb3141-c116-4eaf-86a0-81d1938bb0bf None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:04:56.652600 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-37fb3141-c116-4eaf-86a0-81d1938bb0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:56.753070 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 670/1345] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:04:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:04:56.790774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff70b88-7c22-493f-989a-df9ae5dbb4b2 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:04:56.792382 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-5ff70b88-7c22-493f-989a-df9ae5dbb4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:56.881703 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:04:56.901310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff70b88-7c22-493f-989a-df9ae5dbb4b2 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:04:56.977243 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b753335-9224-47fb-8c0d-cff1a30ff6f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:04:56.977872 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b753335-9224-47fb-8c0d-cff1a30ff6f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:04:56.978156 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b753335-9224-47fb-8c0d-cff1a30ff6f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:04:56.978438 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b753335-9224-47fb-8c0d-cff1a30ff6f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:04:56.980121 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-1b753335-9224-47fb-8c0d-cff1a30ff6f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:04:57.091729 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b753335-9224-47fb-8c0d-cff1a30ff6f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:04:57.093213 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b753335-9224-47fb-8c0d-cff1a30ff6f2 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:04:57.114602 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 676/1346] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:04:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:05:01.571146 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a1cfd-d0c5-4b67-af5b-4c6237c5fcd9 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:01.571642 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a1cfd-d0c5-4b67-af5b-4c6237c5fcd9 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:01.571741 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a1cfd-d0c5-4b67-af5b-4c6237c5fcd9 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:01.573138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 671/1347] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:05:01.578700 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027646f5-e866-44dd-a6ab-ada34f751a64 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:01.579233 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027646f5-e866-44dd-a6ab-ada34f751a64 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:01.579332 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027646f5-e866-44dd-a6ab-ada34f751a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:01.595457 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:01.655584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-027646f5-e866-44dd-a6ab-ada34f751a64 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:05:01.656832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-027646f5-e866-44dd-a6ab-ada34f751a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:01.763563 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 677/1348] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:05:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:05:01.804061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a1cfd-d0c5-4b67-af5b-4c6237c5fcd9 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:05:01.805725 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-053a1cfd-d0c5-4b67-af5b-4c6237c5fcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:01.872826 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:01.928249 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a1cfd-d0c5-4b67-af5b-4c6237c5fcd9 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:05:02.007018 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd771a-3dbd-4501-b248-52dc470e12e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:05:02.007667 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd771a-3dbd-4501-b248-52dc470e12e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:02.008003 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd771a-3dbd-4501-b248-52dc470e12e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:02.008313 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd771a-3dbd-4501-b248-52dc470e12e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:02.009951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-12bd771a-3dbd-4501-b248-52dc470e12e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:02.121400 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:02.123816 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd771a-3dbd-4501-b248-52dc470e12e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:05:02.125263 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd771a-3dbd-4501-b248-52dc470e12e5 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:05:02.143943 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 672/1349] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:05:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:05:07.160868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878dbe1c-a807-4e3b-a38f-93a85f83f921 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:07.161366 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878dbe1c-a807-4e3b-a38f-93a85f83f921 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:07.161474 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878dbe1c-a807-4e3b-a38f-93a85f83f921 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:07.164222 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 678/1350] 10.222.0.40 () {54 vars in 921 bytes} [Wed Aug 3 05:05:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:05:07.178497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee9206-80ee-4211-86c4-2e1168b65e6d None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:07.181230 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee9206-80ee-4211-86c4-2e1168b65e6d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:07.183349 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee9206-80ee-4211-86c4-2e1168b65e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:07.313291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-18ee9206-80ee-4211-86c4-2e1168b65e6d None None] MFA Rules not processed for user `59846d88f4314c2382aa5cb8bae5053b`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:05:07.314886 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-18ee9206-80ee-4211-86c4-2e1168b65e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:07.449311 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 673/1351] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Aug 3 05:05:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:05:11.622255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67f93d-3aff-4a99-9ded-79cede6257b5 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:11.622827 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67f93d-3aff-4a99-9ded-79cede6257b5 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:11.622827 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67f93d-3aff-4a99-9ded-79cede6257b5 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:11.624664 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 679/1352] 10.222.0.40 () {54 vars in 921 bytes} [Wed Aug 3 05:05:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:05:11.630281 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90f451-1c7d-48f2-b6ac-19b5d340009f None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:11.630756 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90f451-1c7d-48f2-b6ac-19b5d340009f None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:11.630899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90f451-1c7d-48f2-b6ac-19b5d340009f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:11.687303 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:11.712447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-cf90f451-1c7d-48f2-b6ac-19b5d340009f None None] MFA Rules not processed for user `59846d88f4314c2382aa5cb8bae5053b`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:05:11.713702 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-cf90f451-1c7d-48f2-b6ac-19b5d340009f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:11.761576 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=59 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:11.840354 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 674/1353] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Aug 3 05:05:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:05:13.095217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff10b90-0596-4b9b-baa0-ff2200bcd38f None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:13.095767 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff10b90-0596-4b9b-baa0-ff2200bcd38f None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:13.095767 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff10b90-0596-4b9b-baa0-ff2200bcd38f None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:13.097116 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 680/1354] 10.222.0.40 () {54 vars in 921 bytes} [Wed Aug 3 05:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:05:13.102437 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b43bed-1b76-422a-87ff-1beaad8652e1 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:13.102886 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b43bed-1b76-422a-87ff-1beaad8652e1 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:13.103003 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b43bed-1b76-422a-87ff-1beaad8652e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:13.180784 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-56b43bed-1b76-422a-87ff-1beaad8652e1 None None] MFA Rules not processed for user `59846d88f4314c2382aa5cb8bae5053b`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:05:13.181993 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-56b43bed-1b76-422a-87ff-1beaad8652e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:13.225224 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:13.296543 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 675/1355] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Aug 3 05:05:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:05:16.696216 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b552394-097f-4542-aab7-fca2f30ff11a None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:16.697103 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b552394-097f-4542-aab7-fca2f30ff11a None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:16.697544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b552394-097f-4542-aab7-fca2f30ff11a None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:16.703051 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 681/1356] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:05:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:05:16.704607 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b2d18-f014-4820-991c-3ec715509aa0 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:16.707288 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b2d18-f014-4820-991c-3ec715509aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:16.707842 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b2d18-f014-4820-991c-3ec715509aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:16.796024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-6d0b2d18-f014-4820-991c-3ec715509aa0 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:05:16.797375 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6d0b2d18-f014-4820-991c-3ec715509aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:16.902998 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 676/1357] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:05:16.956691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279bbdb-c763-42db-9a94-71ce5f8bed92 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:16.957641 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279bbdb-c763-42db-9a94-71ce5f8bed92 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:16.958367 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279bbdb-c763-42db-9a94-71ce5f8bed92 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:16.962123 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 682/1358] 10.222.0.40 () {54 vars in 962 bytes} [Wed Aug 3 05:05:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:05:16.970615 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722d5ff-0f7a-4da2-8482-63b013d0c72b None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:16.973361 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722d5ff-0f7a-4da2-8482-63b013d0c72b None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:16.974460 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722d5ff-0f7a-4da2-8482-63b013d0c72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:16.997367 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:17.062884 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-3722d5ff-0f7a-4da2-8482-63b013d0c72b None None] MFA Rules not processed for user `a3fa4a6c16274c18849869b55b33ac62`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:05:17.064083 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3722d5ff-0f7a-4da2-8482-63b013d0c72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:17.188912 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 677/1359] 10.222.0.40 () {58 vars in 1059 bytes} [Wed Aug 3 05:05:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:05:17.197256 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d279bbdb-c763-42db-9a94-71ce5f8bed92 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:05:17.199177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-d279bbdb-c763-42db-9a94-71ce5f8bed92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:17.372509 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d279bbdb-c763-42db-9a94-71ce5f8bed92 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:05:17.452254 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:17.495959 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2621d-d992-4e4d-8eb4-3d338427ad41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:05:17.496777 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2621d-d992-4e4d-8eb4-3d338427ad41 service nova] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:17.497167 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2621d-d992-4e4d-8eb4-3d338427ad41 service nova] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:17.497529 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2621d-d992-4e4d-8eb4-3d338427ad41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:17.499327 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-64a2621d-d992-4e4d-8eb4-3d338427ad41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:17.601387 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a2621d-d992-4e4d-8eb4-3d338427ad41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:05:17.603280 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a2621d-d992-4e4d-8eb4-3d338427ad41 service nova] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:05:17.631218 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 683/1360] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:05:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:05:19.866392 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef203d7-3fec-494c-b21c-e3a11414a27d None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:19.867403 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef203d7-3fec-494c-b21c-e3a11414a27d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:19.867700 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef203d7-3fec-494c-b21c-e3a11414a27d None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:19.869563 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 678/1361] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:05:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:05:19.877203 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104622e-9076-4b56-9416-45c60e2cf34f None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:19.879808 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104622e-9076-4b56-9416-45c60e2cf34f None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:19.879808 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104622e-9076-4b56-9416-45c60e2cf34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:19.949761 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-3104622e-9076-4b56-9416-45c60e2cf34f None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:05:19.951150 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-3104622e-9076-4b56-9416-45c60e2cf34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:19.990026 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:20.052661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 684/1362] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:05:20.102243 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef203d7-3fec-494c-b21c-e3a11414a27d None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:05:20.103936 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9ef203d7-3fec-494c-b21c-e3a11414a27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:20.191241 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:20.284178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef203d7-3fec-494c-b21c-e3a11414a27d None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:05:20.423101 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68bcdc4-8529-4e11-92d2-c437f75597f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:05:20.423796 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bcdc4-8529-4e11-92d2-c437f75597f9 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:20.424145 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bcdc4-8529-4e11-92d2-c437f75597f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:20.424489 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bcdc4-8529-4e11-92d2-c437f75597f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:20.426607 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c68bcdc4-8529-4e11-92d2-c437f75597f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:20.452414 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:20.525871 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68bcdc4-8529-4e11-92d2-c437f75597f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:05:20.527607 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68bcdc4-8529-4e11-92d2-c437f75597f9 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:05:20.546475 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 679/1363] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:05:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:05:23.030004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805d3cc-c82d-40c6-aa05-aa0a56dfb167 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:23.030534 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805d3cc-c82d-40c6-aa05-aa0a56dfb167 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:23.030534 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805d3cc-c82d-40c6-aa05-aa0a56dfb167 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:23.031924 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 685/1364] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:05:23.037325 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca47ab-92e7-4d5c-9f77-9ac634f2fcd0 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:23.037761 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca47ab-92e7-4d5c-9f77-9ac634f2fcd0 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:23.037950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca47ab-92e7-4d5c-9f77-9ac634f2fcd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:23.110461 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-d4ca47ab-92e7-4d5c-9f77-9ac634f2fcd0 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:05:23.111705 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d4ca47ab-92e7-4d5c-9f77-9ac634f2fcd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:23.220126 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:23.223628 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 680/1365] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:05:23.273103 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585eef15-1f54-4c6a-a546-1491de569229 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:23.273399 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585eef15-1f54-4c6a-a546-1491de569229 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:23.274017 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585eef15-1f54-4c6a-a546-1491de569229 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:23.275538 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 686/1366] 10.222.0.40 () {54 vars in 962 bytes} [Wed Aug 3 05:05:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:05:23.281083 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595fdaf1-88ca-4078-a248-5f2d20089df4 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:23.281315 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595fdaf1-88ca-4078-a248-5f2d20089df4 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:23.281581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595fdaf1-88ca-4078-a248-5f2d20089df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:23.378652 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-595fdaf1-88ca-4078-a248-5f2d20089df4 None None] MFA Rules not processed for user `a3fa4a6c16274c18849869b55b33ac62`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:05:23.379887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-595fdaf1-88ca-4078-a248-5f2d20089df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:23.512146 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 681/1367] 10.222.0.40 () {58 vars in 1059 bytes} [Wed Aug 3 05:05:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:05:23.520237 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585eef15-1f54-4c6a-a546-1491de569229 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:05:23.521756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-585eef15-1f54-4c6a-a546-1491de569229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:23.714933 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585eef15-1f54-4c6a-a546-1491de569229 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:05:23.737171 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:23.836654 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca9744e-1e10-449b-a705-1f4c3df07882 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:05:23.837488 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9744e-1e10-449b-a705-1f4c3df07882 service nova] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:23.837861 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9744e-1e10-449b-a705-1f4c3df07882 service nova] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:23.838238 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9744e-1e10-449b-a705-1f4c3df07882 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:23.840017 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-cca9744e-1e10-449b-a705-1f4c3df07882 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:23.937342 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca9744e-1e10-449b-a705-1f4c3df07882 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:05:23.939160 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca9744e-1e10-449b-a705-1f4c3df07882 service nova] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:05:23.961237 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 687/1368] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:05:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:05:24.925753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48274852-c125-4a8b-830d-cbe10fd5f5c6 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:24.926305 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48274852-c125-4a8b-830d-cbe10fd5f5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:24.926305 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48274852-c125-4a8b-830d-cbe10fd5f5c6 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:24.927631 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 682/1369] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:05:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:05:24.933814 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eea954-0fee-45ac-a7e8-63e65224f0ae None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:24.934289 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eea954-0fee-45ac-a7e8-63e65224f0ae None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:24.934574 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eea954-0fee-45ac-a7e8-63e65224f0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:24.968424 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:25.004611 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-84eea954-0fee-45ac-a7e8-63e65224f0ae None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:05:25.006397 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-84eea954-0fee-45ac-a7e8-63e65224f0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:25.112655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 688/1370] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:05:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:05:25.461063 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1387c57-faa7-464c-9f65-363833eb52fa None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:25.461353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1387c57-faa7-464c-9f65-363833eb52fa None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:25.461607 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1387c57-faa7-464c-9f65-363833eb52fa None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:25.466036 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 683/1371] 10.222.0.40 () {54 vars in 964 bytes} [Wed Aug 3 05:05:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:05:25.469241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25608bc3-624a-4ff3-8c6b-fe3cf81a84de None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:25.469533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25608bc3-624a-4ff3-8c6b-fe3cf81a84de None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:25.469812 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25608bc3-624a-4ff3-8c6b-fe3cf81a84de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:25.552215 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-25608bc3-624a-4ff3-8c6b-fe3cf81a84de None None] MFA Rules not processed for user `c81ca55d52244bf1bfe4e829b6880c20`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:05:25.553629 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-25608bc3-624a-4ff3-8c6b-fe3cf81a84de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:25.604008 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:25.635118 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 689/1372] 10.222.0.40 () {58 vars in 1061 bytes} [Wed Aug 3 05:05:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:05:25.643197 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1387c57-faa7-464c-9f65-363833eb52fa None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:05:25.644891 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f1387c57-faa7-464c-9f65-363833eb52fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:25.705451 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:25.764021 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1387c57-faa7-464c-9f65-363833eb52fa None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:05:25.844697 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45bb68-d43a-4238-9719-b13498b2ec6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:05:25.845350 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45bb68-d43a-4238-9719-b13498b2ec6d service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:25.845679 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45bb68-d43a-4238-9719-b13498b2ec6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:25.845960 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45bb68-d43a-4238-9719-b13498b2ec6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:25.847657 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0d45bb68-d43a-4238-9719-b13498b2ec6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:25.961914 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:25.977113 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d45bb68-d43a-4238-9719-b13498b2ec6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:05:25.978617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d45bb68-d43a-4238-9719-b13498b2ec6d service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:05:25.998479 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 684/1373] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:05:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:05:26.213359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f0383-405d-4504-b23c-481585ad64e5 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:26.213799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f0383-405d-4504-b23c-481585ad64e5 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:26.214176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f0383-405d-4504-b23c-481585ad64e5 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:26.216848 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 690/1374] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:05:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:05:26.221792 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575aaac-ed72-473f-b440-88f28d2417a5 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:26.221970 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575aaac-ed72-473f-b440-88f28d2417a5 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:26.222241 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575aaac-ed72-473f-b440-88f28d2417a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:26.320528 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-2575aaac-ed72-473f-b440-88f28d2417a5 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:05:26.322667 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2575aaac-ed72-473f-b440-88f28d2417a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:26.449706 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 685/1375] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:05:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:05:26.498499 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f0383-405d-4504-b23c-481585ad64e5 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:05:26.500095 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-e41f0383-405d-4504-b23c-481585ad64e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:26.650368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f0383-405d-4504-b23c-481585ad64e5 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:05:26.704128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:26.768100 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250dc35f-fdbd-428d-8dff-2d6719ca1b98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:05:26.768714 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250dc35f-fdbd-428d-8dff-2d6719ca1b98 service nova] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:26.768993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250dc35f-fdbd-428d-8dff-2d6719ca1b98 service nova] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:26.769266 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250dc35f-fdbd-428d-8dff-2d6719ca1b98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:26.770892 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-250dc35f-fdbd-428d-8dff-2d6719ca1b98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:26.861797 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250dc35f-fdbd-428d-8dff-2d6719ca1b98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:05:26.863548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250dc35f-fdbd-428d-8dff-2d6719ca1b98 service nova] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:05:26.882524 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 691/1376] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:05:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:05:29.061578 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca5e6a-27f5-4c88-b940-1e02e75eaad0 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:29.062124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca5e6a-27f5-4c88-b940-1e02e75eaad0 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:29.062224 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca5e6a-27f5-4c88-b940-1e02e75eaad0 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:29.063598 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 686/1377] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:05:29.069013 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcc70f-d0be-4e1e-9c92-3824f95ca8d0 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:29.069422 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcc70f-d0be-4e1e-9c92-3824f95ca8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:29.069576 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcc70f-d0be-4e1e-9c92-3824f95ca8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:29.154739 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-6dfcc70f-d0be-4e1e-9c92-3824f95ca8d0 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:05:29.156407 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-6dfcc70f-d0be-4e1e-9c92-3824f95ca8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:29.163201 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:29.254287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 692/1378] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:05:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:05:29.297967 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca5e6a-27f5-4c88-b940-1e02e75eaad0 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:05:29.299497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ffca5e6a-27f5-4c88-b940-1e02e75eaad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:29.362299 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:29.452047 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca5e6a-27f5-4c88-b940-1e02e75eaad0 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:05:29.609503 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbf9fa-5f93-45ee-9ed3-7ece3d8a09d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:05:29.610902 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbf9fa-5f93-45ee-9ed3-7ece3d8a09d3 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:29.610902 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbf9fa-5f93-45ee-9ed3-7ece3d8a09d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:29.610902 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbf9fa-5f93-45ee-9ed3-7ece3d8a09d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:29.614567 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-bfbbf9fa-5f93-45ee-9ed3-7ece3d8a09d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:29.644879 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:29.740373 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbbf9fa-5f93-45ee-9ed3-7ece3d8a09d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:05:29.742651 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbbf9fa-5f93-45ee-9ed3-7ece3d8a09d3 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:05:29.763323 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 687/1379] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:05:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:05:31.933719 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a8518-51ec-438a-922c-7bf985d0b80e None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:31.934200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a8518-51ec-438a-922c-7bf985d0b80e None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:31.934519 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a8518-51ec-438a-922c-7bf985d0b80e None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:31.935948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 693/1380] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:05:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:05:31.941558 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a7dda-4ad4-4512-a775-e923f2d18af8 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:31.941964 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a7dda-4ad4-4512-a775-e923f2d18af8 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:31.942133 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a7dda-4ad4-4512-a775-e923f2d18af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:32.017929 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-bc3a7dda-4ad4-4512-a775-e923f2d18af8 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:05:32.019180 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-bc3a7dda-4ad4-4512-a775-e923f2d18af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:32.116044 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:32.123017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 688/1381] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:05:32.168261 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a8518-51ec-438a-922c-7bf985d0b80e None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:05:32.169769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-a62a8518-51ec-438a-922c-7bf985d0b80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:32.319725 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:32.331489 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a8518-51ec-438a-922c-7bf985d0b80e None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:05:32.464418 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c763cf-bffc-411b-b447-06beee11fe9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:05:32.465060 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c763cf-bffc-411b-b447-06beee11fe9b service nova] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:32.465347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c763cf-bffc-411b-b447-06beee11fe9b service nova] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:32.465626 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c763cf-bffc-411b-b447-06beee11fe9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:32.467491 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-e6c763cf-bffc-411b-b447-06beee11fe9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:32.570804 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c763cf-bffc-411b-b447-06beee11fe9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:05:32.572458 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c763cf-bffc-411b-b447-06beee11fe9b service nova] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:05:32.583467 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:32.592076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 694/1382] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:05:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:05:34.804968 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5629b8-d149-4574-827f-fb28e88d0c9d None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:34.805524 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5629b8-d149-4574-827f-fb28e88d0c9d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:34.805524 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5629b8-d149-4574-827f-fb28e88d0c9d None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:34.806988 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 689/1383] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:05:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:05:34.812568 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839f98e-0011-4526-82ed-ab2375e1b868 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:34.813041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839f98e-0011-4526-82ed-ab2375e1b868 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:34.813143 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839f98e-0011-4526-82ed-ab2375e1b868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:34.882641 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-6839f98e-0011-4526-82ed-ab2375e1b868 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:05:34.883919 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-6839f98e-0011-4526-82ed-ab2375e1b868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:34.987423 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 695/1384] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:05:35.032419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5629b8-d149-4574-827f-fb28e88d0c9d None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:05:35.034024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ce5629b8-d149-4574-827f-fb28e88d0c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:35.196883 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5629b8-d149-4574-827f-fb28e88d0c9d None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:05:35.269079 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:35.321476 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dabf6f-77ab-4e19-8d81-085d8a627510 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:05:35.322106 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dabf6f-77ab-4e19-8d81-085d8a627510 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:35.322395 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dabf6f-77ab-4e19-8d81-085d8a627510 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:35.322674 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dabf6f-77ab-4e19-8d81-085d8a627510 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:35.324342 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e1dabf6f-77ab-4e19-8d81-085d8a627510 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:35.420539 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dabf6f-77ab-4e19-8d81-085d8a627510 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:05:35.422600 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dabf6f-77ab-4e19-8d81-085d8a627510 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:05:35.446276 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 690/1385] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:05:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:05:38.011577 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec5bbe-8505-4325-b185-2b59bf279583 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:38.012115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec5bbe-8505-4325-b185-2b59bf279583 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:38.012264 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec5bbe-8505-4325-b185-2b59bf279583 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:38.013874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 696/1386] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:05:38.020639 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2b66c-ffe2-4ef3-a185-40cba60c67dc None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:38.021077 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2b66c-ffe2-4ef3-a185-40cba60c67dc None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:38.021345 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2b66c-ffe2-4ef3-a185-40cba60c67dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:38.109023 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-e8f2b66c-ffe2-4ef3-a185-40cba60c67dc None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:05:38.110351 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e8f2b66c-ffe2-4ef3-a185-40cba60c67dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:38.149991 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:38.218619 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 691/1387] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:05:38.266307 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec5bbe-8505-4325-b185-2b59bf279583 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:05:38.267933 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-c1ec5bbe-8505-4325-b185-2b59bf279583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:38.347757 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:38.424813 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec5bbe-8505-4325-b185-2b59bf279583 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:05:38.542122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11534f90-2574-4d78-94f0-865f8c1618d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:05:38.542747 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11534f90-2574-4d78-94f0-865f8c1618d1 service nova] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:38.543058 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11534f90-2574-4d78-94f0-865f8c1618d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:38.543334 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11534f90-2574-4d78-94f0-865f8c1618d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:38.544967 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-11534f90-2574-4d78-94f0-865f8c1618d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:38.584521 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:38.639087 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11534f90-2574-4d78-94f0-865f8c1618d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:05:38.640776 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11534f90-2574-4d78-94f0-865f8c1618d1 service nova] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:05:38.661152 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 697/1388] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:05:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:05:41.023848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75177c-4513-4fc7-8fd7-777048ee24f7 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:41.024441 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75177c-4513-4fc7-8fd7-777048ee24f7 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:41.024441 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75177c-4513-4fc7-8fd7-777048ee24f7 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:41.025774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 692/1389] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:05:41.031070 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258cd74b-72c1-4ab3-a49a-a6954f470daa None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:41.031510 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258cd74b-72c1-4ab3-a49a-a6954f470daa None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:41.031614 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258cd74b-72c1-4ab3-a49a-a6954f470daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:41.101040 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-258cd74b-72c1-4ab3-a49a-a6954f470daa None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:05:41.102612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-258cd74b-72c1-4ab3-a49a-a6954f470daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:41.203376 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 698/1390] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:05:41.248821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f75177c-4513-4fc7-8fd7-777048ee24f7 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:05:41.250323 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2f75177c-4513-4fc7-8fd7-777048ee24f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:41.395158 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f75177c-4513-4fc7-8fd7-777048ee24f7 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:05:41.411362 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:41.438510 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdce9a9-2d68-4d5c-8f91-b86571613ada None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:41.438801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdce9a9-2d68-4d5c-8f91-b86571613ada None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:41.439112 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdce9a9-2d68-4d5c-8f91-b86571613ada None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:41.440567 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 699/1391] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 05:05:41.445604 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed1094a-cd70-4d83-9d24-47e5e63ec457 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:41.445891 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed1094a-cd70-4d83-9d24-47e5e63ec457 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:41.446172 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed1094a-cd70-4d83-9d24-47e5e63ec457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:41.461874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:41.510236 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee908f12-cf11-4701-8e0a-be0e69f116e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:05:41.510881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee908f12-cf11-4701-8e0a-be0e69f116e8 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:41.511169 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee908f12-cf11-4701-8e0a-be0e69f116e8 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:41.511446 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee908f12-cf11-4701-8e0a-be0e69f116e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:41.513086 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ee908f12-cf11-4701-8e0a-be0e69f116e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:41.514928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-6ed1094a-cd70-4d83-9d24-47e5e63ec457 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:05:41.516130 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-6ed1094a-cd70-4d83-9d24-47e5e63ec457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:41.607859 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee908f12-cf11-4701-8e0a-be0e69f116e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:05:41.609535 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee908f12-cf11-4701-8e0a-be0e69f116e8 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:05:41.616799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 700/1392] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:05:41.630743 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 693/1393] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:05:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:05:41.679174 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08887711-406b-4290-bef7-a1d5c9007042 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:41.682465 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08887711-406b-4290-bef7-a1d5c9007042 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:41.682465 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08887711-406b-4290-bef7-a1d5c9007042 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:41.682465 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 701/1394] 10.222.0.40 () {54 vars in 966 bytes} [Wed Aug 3 05:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:05:41.686321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396886a-423d-492e-ba8c-1d3d4f2dac6d None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:41.686613 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396886a-423d-492e-ba8c-1d3d4f2dac6d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:41.687038 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396886a-423d-492e-ba8c-1d3d4f2dac6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:41.715655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:41.767330 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-0396886a-423d-492e-ba8c-1d3d4f2dac6d None None] MFA Rules not processed for user `84cc14b33895443b893fab487bfe1858`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:05:41.768614 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0396886a-423d-492e-ba8c-1d3d4f2dac6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:41.856770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 694/1395] 10.222.0.40 () {58 vars in 1063 bytes} [Wed Aug 3 05:05:41 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:05:41.864526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08887711-406b-4290-bef7-a1d5c9007042 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:05:41.866247 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-08887711-406b-4290-bef7-a1d5c9007042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:41.971190 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:41.978246 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08887711-406b-4290-bef7-a1d5c9007042 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:05:42.056120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef444541-2688-4cd1-9679-4097cfcaaf0b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cc14b33895443b893fab487bfe1858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:05:42.056770 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef444541-2688-4cd1-9679-4097cfcaaf0b service glance] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:42.057010 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef444541-2688-4cd1-9679-4097cfcaaf0b service glance] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:42.057283 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef444541-2688-4cd1-9679-4097cfcaaf0b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:42.058946 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-ef444541-2688-4cd1-9679-4097cfcaaf0b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:42.173361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef444541-2688-4cd1-9679-4097cfcaaf0b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:05:42.174805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef444541-2688-4cd1-9679-4097cfcaaf0b service glance] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:05:42.194902 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 702/1396] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:05:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:05:43.081736 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983d370-4fc8-42d4-a367-314d4859cb7a None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:43.082204 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983d370-4fc8-42d4-a367-314d4859cb7a None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:43.082321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983d370-4fc8-42d4-a367-314d4859cb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:43.151274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-1983d370-4fc8-42d4-a367-314d4859cb7a None None] MFA Rules not processed for user `de853a2dfffb4f05b6f47df2280ae4e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:05:43.152988 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1983d370-4fc8-42d4-a367-314d4859cb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:43.215899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:43.249618 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 695/1397] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Aug 3 05:05:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:05:43.289065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db943a8-c682-46b2-90f4-bb2cef7166d5 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:43.289584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db943a8-c682-46b2-90f4-bb2cef7166d5 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:43.289691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db943a8-c682-46b2-90f4-bb2cef7166d5 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:43.291227 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 703/1398] 10.222.0.40 () {54 vars in 943 bytes} [Wed Aug 3 05:05:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:05:43.297572 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936fa4e-4a72-4ad1-9e2b-2c5749c62908 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:43.297927 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936fa4e-4a72-4ad1-9e2b-2c5749c62908 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:43.298210 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936fa4e-4a72-4ad1-9e2b-2c5749c62908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:43.377238 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-d936fa4e-4a72-4ad1-9e2b-2c5749c62908 None None] MFA Rules not processed for user `e1014bcb4d36441aa46042941c9a63d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:05:43.378372 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d936fa4e-4a72-4ad1-9e2b-2c5749c62908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:43.486881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 696/1399] 10.222.0.40 () {58 vars in 1040 bytes} [Wed Aug 3 05:05:43 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:05:43.496761 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db943a8-c682-46b2-90f4-bb2cef7166d5 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:05:43.498316 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-6db943a8-c682-46b2-90f4-bb2cef7166d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:43.523591 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:43.640485 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db943a8-c682-46b2-90f4-bb2cef7166d5 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:05:43.757617 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a172b-bd6d-4c08-af8e-5e5dff328e20 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1014bcb4d36441aa46042941c9a63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:05:43.758236 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a172b-bd6d-4c08-af8e-5e5dff328e20 service swift] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:43.758517 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a172b-bd6d-4c08-af8e-5e5dff328e20 service swift] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:43.758789 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a172b-bd6d-4c08-af8e-5e5dff328e20 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:43.760465 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-928a172b-bd6d-4c08-af8e-5e5dff328e20 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:43.767636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:43.789468 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fea02-743a-4bb7-842a-22f130ea2071 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:43.789753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fea02-743a-4bb7-842a-22f130ea2071 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:43.790026 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fea02-743a-4bb7-842a-22f130ea2071 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:43.791418 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 697/1400] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 05:05:43.796405 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6940a6b-3518-4ccb-b831-76916ae8f95f None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:43.796686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6940a6b-3518-4ccb-b831-76916ae8f95f None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:43.796959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6940a6b-3518-4ccb-b831-76916ae8f95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:43.842421 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:43.846618 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a172b-bd6d-4c08-af8e-5e5dff328e20 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:05:43.848289 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a172b-bd6d-4c08-af8e-5e5dff328e20 service swift] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:05:43.856600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 704/1401] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 3 05:05:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 05:05:43.864549 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-b6940a6b-3518-4ccb-b831-76916ae8f95f None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:05:43.866347 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b6940a6b-3518-4ccb-b831-76916ae8f95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:43.966984 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 698/1402] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:05:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:05:44.011417 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a172b-bd6d-4c08-af8e-5e5dff328e20 service swift] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:05:44.012646 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-928a172b-bd6d-4c08-af8e-5e5dff328e20 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:44.106534 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a172b-bd6d-4c08-af8e-5e5dff328e20 service swift] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:05:44.166909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:44.233214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7fc9b-125e-4318-a29e-cd1dc24afd9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:05:44.233750 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7fc9b-125e-4318-a29e-cd1dc24afd9b service nova] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:44.234034 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7fc9b-125e-4318-a29e-cd1dc24afd9b service nova] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:44.234309 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7fc9b-125e-4318-a29e-cd1dc24afd9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:44.235976 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-e7e7fc9b-125e-4318-a29e-cd1dc24afd9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:44.327504 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e7fc9b-125e-4318-a29e-cd1dc24afd9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:05:44.329127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e7fc9b-125e-4318-a29e-cd1dc24afd9b service nova] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:05:44.346780 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 705/1403] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:05:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:05:46.677027 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ae112-c1eb-42a7-92fd-6b0ea447ae0f None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:46.677549 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ae112-c1eb-42a7-92fd-6b0ea447ae0f None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:46.677642 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ae112-c1eb-42a7-92fd-6b0ea447ae0f None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:46.679733 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 699/1404] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:05:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:05:46.684844 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f8906-9bfa-4d2d-889a-1719780d93d1 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:46.685272 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f8906-9bfa-4d2d-889a-1719780d93d1 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:46.685502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f8906-9bfa-4d2d-889a-1719780d93d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:46.755088 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-064f8906-9bfa-4d2d-889a-1719780d93d1 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:05:46.757506 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-064f8906-9bfa-4d2d-889a-1719780d93d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:46.771158 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:46.877373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 706/1405] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:05:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:05:46.924241 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731ae112-c1eb-42a7-92fd-6b0ea447ae0f None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:05:46.925792 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-731ae112-c1eb-42a7-92fd-6b0ea447ae0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:47.067007 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:47.104519 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731ae112-c1eb-42a7-92fd-6b0ea447ae0f None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:05:47.251517 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff740a-89e1-4afa-b6ee-33eaa5663baa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:05:47.252159 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff740a-89e1-4afa-b6ee-33eaa5663baa service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:47.252440 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff740a-89e1-4afa-b6ee-33eaa5663baa service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:47.252717 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff740a-89e1-4afa-b6ee-33eaa5663baa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:47.254475 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-69ff740a-89e1-4afa-b6ee-33eaa5663baa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:47.346801 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:47.361478 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff740a-89e1-4afa-b6ee-33eaa5663baa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:05:47.363121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff740a-89e1-4afa-b6ee-33eaa5663baa service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:05:47.385759 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 700/1406] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:05:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:05:50.274385 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9d4c6-fed7-47c1-87e6-2195e3d5308b None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:50.275030 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9d4c6-fed7-47c1-87e6-2195e3d5308b None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:50.275030 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9d4c6-fed7-47c1-87e6-2195e3d5308b None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:50.279204 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 707/1407] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:05:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:05:50.281536 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f55632-2ef6-4466-9ec1-71fce15fc6c5 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:50.284184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f55632-2ef6-4466-9ec1-71fce15fc6c5 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:50.284755 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f55632-2ef6-4466-9ec1-71fce15fc6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:50.362599 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-c3f55632-2ef6-4466-9ec1-71fce15fc6c5 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:05:50.364017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c3f55632-2ef6-4466-9ec1-71fce15fc6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:50.477535 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 701/1408] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:05:50.527282 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9d4c6-fed7-47c1-87e6-2195e3d5308b None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:05:50.529117 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-a8f9d4c6-fed7-47c1-87e6-2195e3d5308b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:50.704407 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:05:50.707860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9d4c6-fed7-47c1-87e6-2195e3d5308b None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:05:50.846774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521e26d-42bb-4b3d-80c7-b21b68b840c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:05:50.847357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521e26d-42bb-4b3d-80c7-b21b68b840c7 service nova] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:05:50.847655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521e26d-42bb-4b3d-80c7-b21b68b840c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:05:50.847945 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521e26d-42bb-4b3d-80c7-b21b68b840c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:05:50.850172 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-4521e26d-42bb-4b3d-80c7-b21b68b840c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:05:50.961916 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4521e26d-42bb-4b3d-80c7-b21b68b840c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:05:50.964209 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4521e26d-42bb-4b3d-80c7-b21b68b840c7 service nova] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:05:50.984232 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 708/1409] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:05:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:06:00.475737 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=76 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:00.981743 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:07.965906 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774614d5-db27-4663-8421-b0a81eb7e8ac None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:07.966443 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774614d5-db27-4663-8421-b0a81eb7e8ac None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:07.966443 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774614d5-db27-4663-8421-b0a81eb7e8ac None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:07.967782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 702/1410] 10.222.0.40 () {54 vars in 918 bytes} [Wed Aug 3 05:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 05:06:07.972990 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7438c-8a37-41b2-a98b-4d7888f7498e None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:07.973418 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7438c-8a37-41b2-a98b-4d7888f7498e None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:07.974153 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7438c-8a37-41b2-a98b-4d7888f7498e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:08.052767 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-b8b7438c-8a37-41b2-a98b-4d7888f7498e None None] MFA Rules not processed for user `59846d88f4314c2382aa5cb8bae5053b`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:06:08.054011 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-b8b7438c-8a37-41b2-a98b-4d7888f7498e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:08.159936 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 709/1411] 10.222.0.40 () {58 vars in 1015 bytes} [Wed Aug 3 05:06:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:06:08.174767 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c370c3-fdfa-48f0-8363-773329a1ad4b None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:08.175092 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c370c3-fdfa-48f0-8363-773329a1ad4b None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:08.175377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c370c3-fdfa-48f0-8363-773329a1ad4b None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:08.176724 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 703/1412] 10.222.0.40 () {54 vars in 918 bytes} [Wed Aug 3 05:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:06:08.181192 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31fa32f-ab79-4e03-a2b5-46bfcf53ae5c None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:08.181485 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31fa32f-ab79-4e03-a2b5-46bfcf53ae5c None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:08.181794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31fa32f-ab79-4e03-a2b5-46bfcf53ae5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:08.255619 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:08.258657 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-f31fa32f-ab79-4e03-a2b5-46bfcf53ae5c None None] MFA Rules not processed for user `a3fa4a6c16274c18849869b55b33ac62`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:06:08.259927 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-f31fa32f-ab79-4e03-a2b5-46bfcf53ae5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:08.375706 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 710/1413] 10.222.0.40 () {58 vars in 1015 bytes} [Wed Aug 3 05:06:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:06:10.913738 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12952c50-05ea-4c2a-a117-ae2e9a6db08b None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:10.914270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12952c50-05ea-4c2a-a117-ae2e9a6db08b None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:10.914365 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12952c50-05ea-4c2a-a117-ae2e9a6db08b None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:10.915808 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 704/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 05:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:06:10.921012 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3be36d-a150-469e-8005-9f43bff2429b None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:10.921467 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3be36d-a150-469e-8005-9f43bff2429b None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:10.921579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3be36d-a150-469e-8005-9f43bff2429b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:10.988489 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-cf3be36d-a150-469e-8005-9f43bff2429b None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:06:10.990217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-cf3be36d-a150-469e-8005-9f43bff2429b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:11.044669 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:11.086847 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 711/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 05:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:06:11.131101 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12952c50-05ea-4c2a-a117-ae2e9a6db08b None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:06:11.132618 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-12952c50-05ea-4c2a-a117-ae2e9a6db08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:11.288771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12952c50-05ea-4c2a-a117-ae2e9a6db08b None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:06:11.362522 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:11.403151 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c8131-9680-418e-bdeb-838bfd61c473 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:06:11.403780 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c8131-9680-418e-bdeb-838bfd61c473 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:11.404096 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c8131-9680-418e-bdeb-838bfd61c473 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:11.404380 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c8131-9680-418e-bdeb-838bfd61c473 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:11.406011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-154c8131-9680-418e-bdeb-838bfd61c473 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:11.496967 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154c8131-9680-418e-bdeb-838bfd61c473 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:06:11.498588 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154c8131-9680-418e-bdeb-838bfd61c473 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:06:11.516422 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 705/1416] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:06:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:06:12.416669 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367581e7-158a-4546-846b-ff595ecf1e90 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:12.417233 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367581e7-158a-4546-846b-ff595ecf1e90 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:12.417233 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367581e7-158a-4546-846b-ff595ecf1e90 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:12.418579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 712/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 05:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:06:12.423433 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060b5c3-c583-42d5-9367-9a2a396d54e7 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:12.423841 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060b5c3-c583-42d5-9367-9a2a396d54e7 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:12.423984 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060b5c3-c583-42d5-9367-9a2a396d54e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:12.501729 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-b060b5c3-c583-42d5-9367-9a2a396d54e7 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:06:12.502998 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b060b5c3-c583-42d5-9367-9a2a396d54e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:12.530265 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:12.600284 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 706/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 05:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:06:12.751980 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367581e7-158a-4546-846b-ff595ecf1e90 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:06:12.753546 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-367581e7-158a-4546-846b-ff595ecf1e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:12.898904 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367581e7-158a-4546-846b-ff595ecf1e90 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:06:12.939354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:13.012539 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a99238-114f-4e73-bd78-73df98fcae92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:06:13.013083 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a99238-114f-4e73-bd78-73df98fcae92 service nova] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:13.013363 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a99238-114f-4e73-bd78-73df98fcae92 service nova] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:13.013634 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a99238-114f-4e73-bd78-73df98fcae92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:13.015282 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-a9a99238-114f-4e73-bd78-73df98fcae92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:13.103961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a99238-114f-4e73-bd78-73df98fcae92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:06:13.105578 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a99238-114f-4e73-bd78-73df98fcae92 service nova] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:06:13.123586 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 713/1419] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:06:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:06:14.014295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e614f-46b6-4544-bffd-1011080f1c1a None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:14.015412 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e614f-46b6-4544-bffd-1011080f1c1a None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:14.015412 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e614f-46b6-4544-bffd-1011080f1c1a None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:14.016357 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 707/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 05:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:06:14.021435 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899df84-614f-4d31-9318-3a9fc730ff8f None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:14.021873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899df84-614f-4d31-9318-3a9fc730ff8f None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:14.022080 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899df84-614f-4d31-9318-3a9fc730ff8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:14.084007 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:14.105517 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-5899df84-614f-4d31-9318-3a9fc730ff8f None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:06:14.106857 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-5899df84-614f-4d31-9318-3a9fc730ff8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:14.203686 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 714/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 05:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:06:14.361995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e614f-46b6-4544-bffd-1011080f1c1a None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:06:14.363450 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8c7e614f-46b6-4544-bffd-1011080f1c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:14.515264 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e614f-46b6-4544-bffd-1011080f1c1a None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:06:14.527145 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:14.635072 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87e015-92be-41e1-8bff-f92b4c5ba49b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:06:14.635606 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87e015-92be-41e1-8bff-f92b4c5ba49b service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:14.635890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87e015-92be-41e1-8bff-f92b4c5ba49b service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:14.636167 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87e015-92be-41e1-8bff-f92b4c5ba49b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:14.637799 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0e87e015-92be-41e1-8bff-f92b4c5ba49b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:14.728541 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87e015-92be-41e1-8bff-f92b4c5ba49b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:06:14.730158 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87e015-92be-41e1-8bff-f92b4c5ba49b service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:06:14.748716 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 708/1422] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:06:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:06:15.603620 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff3478-d254-4b47-9620-439229c35362 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:15.604123 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff3478-d254-4b47-9620-439229c35362 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:15.604222 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff3478-d254-4b47-9620-439229c35362 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:15.605534 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 715/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 05:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:06:15.610776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2570179-0593-4e33-8bf5-bdfe5ec31ce9 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:15.611234 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2570179-0593-4e33-8bf5-bdfe5ec31ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:15.611378 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2570179-0593-4e33-8bf5-bdfe5ec31ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:15.631735 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:15.689300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-d2570179-0593-4e33-8bf5-bdfe5ec31ce9 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:06:15.690490 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d2570179-0593-4e33-8bf5-bdfe5ec31ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:15.787512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 709/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 05:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:06:15.941641 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff3478-d254-4b47-9620-439229c35362 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:06:15.943255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-aaff3478-d254-4b47-9620-439229c35362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:16.053640 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:16.095323 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff3478-d254-4b47-9620-439229c35362 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:06:16.208448 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4b579-2c96-4068-a515-eb90510e138b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:06:16.208981 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4b579-2c96-4068-a515-eb90510e138b service nova] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:16.209259 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4b579-2c96-4068-a515-eb90510e138b service nova] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:16.209551 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4b579-2c96-4068-a515-eb90510e138b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:16.211193 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-d6e4b579-2c96-4068-a515-eb90510e138b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:16.286937 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:16.300276 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e4b579-2c96-4068-a515-eb90510e138b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:06:16.301895 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e4b579-2c96-4068-a515-eb90510e138b service nova] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:06:16.320462 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 716/1425] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:06:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:06:17.190185 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35f204-f726-4c95-8084-f297d1a6fcde None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:17.190750 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35f204-f726-4c95-8084-f297d1a6fcde None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:17.190750 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35f204-f726-4c95-8084-f297d1a6fcde None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:17.192136 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 710/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 05:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:06:17.196998 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86977e-0cc8-4b9b-be6d-12c0fa0c0f9f None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:17.197445 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86977e-0cc8-4b9b-be6d-12c0fa0c0f9f None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:17.197629 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86977e-0cc8-4b9b-be6d-12c0fa0c0f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:17.274568 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-1a86977e-0cc8-4b9b-be6d-12c0fa0c0f9f None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:06:17.275829 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-1a86977e-0cc8-4b9b-be6d-12c0fa0c0f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:17.372919 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 717/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 05:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:06:17.527922 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d35f204-f726-4c95-8084-f297d1a6fcde None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:06:17.529409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7d35f204-f726-4c95-8084-f297d1a6fcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:17.621476 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:17.684257 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d35f204-f726-4c95-8084-f297d1a6fcde None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:06:17.796505 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ee4cf-0bee-4be1-a3a0-003df9439d4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:06:17.797045 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ee4cf-0bee-4be1-a3a0-003df9439d4d service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:17.797329 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ee4cf-0bee-4be1-a3a0-003df9439d4d service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:17.797605 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ee4cf-0bee-4be1-a3a0-003df9439d4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:17.799264 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-732ee4cf-0bee-4be1-a3a0-003df9439d4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:17.847297 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:17.889870 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ee4cf-0bee-4be1-a3a0-003df9439d4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:06:17.891522 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ee4cf-0bee-4be1-a3a0-003df9439d4d service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:06:17.909934 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 711/1428] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:06:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:06:18.794868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe776b5-9d9e-48fe-afd0-643b67964326 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:18.795422 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe776b5-9d9e-48fe-afd0-643b67964326 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:18.795422 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe776b5-9d9e-48fe-afd0-643b67964326 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:18.796792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 718/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 05:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:06:18.802402 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af071fde-007f-4349-9b95-ad979dc90d44 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:18.804155 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af071fde-007f-4349-9b95-ad979dc90d44 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:18.804155 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af071fde-007f-4349-9b95-ad979dc90d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:18.999217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-af071fde-007f-4349-9b95-ad979dc90d44 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:06:19.000445 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-af071fde-007f-4349-9b95-ad979dc90d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:19.096340 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 712/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 05:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:06:19.247693 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe776b5-9d9e-48fe-afd0-643b67964326 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:06:19.249213 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-8fe776b5-9d9e-48fe-afd0-643b67964326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:19.297580 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:19.402255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe776b5-9d9e-48fe-afd0-643b67964326 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:06:19.518415 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2bdfa8-88ad-4dd2-8a55-23cd01f7ccab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:06:19.518980 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2bdfa8-88ad-4dd2-8a55-23cd01f7ccab service nova] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:19.519265 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2bdfa8-88ad-4dd2-8a55-23cd01f7ccab service nova] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:19.519535 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2bdfa8-88ad-4dd2-8a55-23cd01f7ccab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:19.521772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-fe2bdfa8-88ad-4dd2-8a55-23cd01f7ccab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:19.528721 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:19.612906 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2bdfa8-88ad-4dd2-8a55-23cd01f7ccab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:06:19.614540 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2bdfa8-88ad-4dd2-8a55-23cd01f7ccab service nova] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:06:19.637342 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 719/1431] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:06:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:06:20.502647 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656afb61-d686-446b-94b2-8adcc3a05121 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:20.503282 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656afb61-d686-446b-94b2-8adcc3a05121 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:20.503282 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656afb61-d686-446b-94b2-8adcc3a05121 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:20.504602 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 713/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 05:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:06:20.509502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92048119-2e7a-42ed-a596-42d2a476fb83 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:20.509900 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92048119-2e7a-42ed-a596-42d2a476fb83 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:20.510071 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92048119-2e7a-42ed-a596-42d2a476fb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:20.589230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-92048119-2e7a-42ed-a596-42d2a476fb83 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:06:20.590473 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-92048119-2e7a-42ed-a596-42d2a476fb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:20.659515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:20.688798 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 720/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 05:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:06:20.842780 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656afb61-d686-446b-94b2-8adcc3a05121 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:06:20.844230 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-656afb61-d686-446b-94b2-8adcc3a05121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:20.861806 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:20.990058 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656afb61-d686-446b-94b2-8adcc3a05121 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:06:21.077027 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:21.103134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc001a0-7956-4579-ac9b-9005a2535854 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:06:21.103687 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc001a0-7956-4579-ac9b-9005a2535854 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:21.103993 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc001a0-7956-4579-ac9b-9005a2535854 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:21.104271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc001a0-7956-4579-ac9b-9005a2535854 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:21.105942 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ddc001a0-7956-4579-ac9b-9005a2535854 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:21.195450 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc001a0-7956-4579-ac9b-9005a2535854 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:06:21.197046 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc001a0-7956-4579-ac9b-9005a2535854 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:06:21.215328 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 714/1434] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:06:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:06:22.077535 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5addce-87fa-47b3-92d2-69525c477eff None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:22.078082 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5addce-87fa-47b3-92d2-69525c477eff None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:22.078082 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5addce-87fa-47b3-92d2-69525c477eff None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:22.079475 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 721/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 05:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:06:22.085005 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e37a4f-08ad-47dd-a27a-3e8b08a9bbc5 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:22.085456 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e37a4f-08ad-47dd-a27a-3e8b08a9bbc5 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:22.085579 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e37a4f-08ad-47dd-a27a-3e8b08a9bbc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:22.162210 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-c6e37a4f-08ad-47dd-a27a-3e8b08a9bbc5 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:06:22.163442 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c6e37a4f-08ad-47dd-a27a-3e8b08a9bbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:22.203893 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:22.260436 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 715/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 05:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:06:22.411804 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5addce-87fa-47b3-92d2-69525c477eff None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:06:22.413342 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-6f5addce-87fa-47b3-92d2-69525c477eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:22.559619 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5addce-87fa-47b3-92d2-69525c477eff None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:06:22.622733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:22.683052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d269da74-5305-473c-8a7c-b258f7c0f4a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:06:22.683677 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269da74-5305-473c-8a7c-b258f7c0f4a7 service nova] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:22.683961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269da74-5305-473c-8a7c-b258f7c0f4a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:22.684236 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269da74-5305-473c-8a7c-b258f7c0f4a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:22.685851 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-d269da74-5305-473c-8a7c-b258f7c0f4a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:22.775516 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d269da74-5305-473c-8a7c-b258f7c0f4a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:06:22.777149 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d269da74-5305-473c-8a7c-b258f7c0f4a7 service nova] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:06:22.797605 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 722/1437] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:06:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:06:23.666630 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469b328-8286-4900-be16-484c0c0d153d None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:23.667077 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469b328-8286-4900-be16-484c0c0d153d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:23.667235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469b328-8286-4900-be16-484c0c0d153d None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:23.668602 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 716/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 05:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:06:23.673507 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e6c5c-9c18-4ce7-81a3-eab686caf115 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:23.673903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e6c5c-9c18-4ce7-81a3-eab686caf115 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:23.674068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e6c5c-9c18-4ce7-81a3-eab686caf115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:23.750296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-ee9e6c5c-9c18-4ce7-81a3-eab686caf115 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:06:23.751535 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-ee9e6c5c-9c18-4ce7-81a3-eab686caf115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:23.753554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:23.850752 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 723/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 05:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:06:24.003253 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d469b328-8286-4900-be16-484c0c0d153d None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:06:24.005405 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d469b328-8286-4900-be16-484c0c0d153d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:24.149734 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d469b328-8286-4900-be16-484c0c0d153d None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:06:24.186013 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:24.273097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a519fb3c-e5b7-4078-9c71-79a9be058407 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:06:24.273624 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519fb3c-e5b7-4078-9c71-79a9be058407 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:24.273906 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519fb3c-e5b7-4078-9c71-79a9be058407 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:24.274180 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519fb3c-e5b7-4078-9c71-79a9be058407 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:24.275851 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a519fb3c-e5b7-4078-9c71-79a9be058407 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:24.373745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a519fb3c-e5b7-4078-9c71-79a9be058407 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:06:24.375376 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a519fb3c-e5b7-4078-9c71-79a9be058407 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:06:24.393221 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 717/1440] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:06:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:06:26.321789 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4186510b-06b9-48d6-ab95-af93c0f281bf None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:26.322354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4186510b-06b9-48d6-ab95-af93c0f281bf None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:26.322354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4186510b-06b9-48d6-ab95-af93c0f281bf None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:26.323768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 724/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 05:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:06:26.329551 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abcc48-b2a7-45bc-aac5-1452952db3e5 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:26.330004 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abcc48-b2a7-45bc-aac5-1452952db3e5 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:26.330111 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abcc48-b2a7-45bc-aac5-1452952db3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:26.371921 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:26.398539 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-84abcc48-b2a7-45bc-aac5-1452952db3e5 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:06:26.400438 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-84abcc48-b2a7-45bc-aac5-1452952db3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:26.497556 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 718/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 05:06:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:06:26.536136 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4186510b-06b9-48d6-ab95-af93c0f281bf None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:06:26.537697 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-4186510b-06b9-48d6-ab95-af93c0f281bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:26.656648 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4186510b-06b9-48d6-ab95-af93c0f281bf None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:06:26.689081 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:26.730993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9463d83-57ac-40db-bd0f-efd650ec2a65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:06:26.731619 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9463d83-57ac-40db-bd0f-efd650ec2a65 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:26.731906 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9463d83-57ac-40db-bd0f-efd650ec2a65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:26.732182 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9463d83-57ac-40db-bd0f-efd650ec2a65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:26.733819 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-f9463d83-57ac-40db-bd0f-efd650ec2a65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:26.841380 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9463d83-57ac-40db-bd0f-efd650ec2a65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:06:26.842825 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9463d83-57ac-40db-bd0f-efd650ec2a65 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:06:26.861812 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 725/1443] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:06:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:06:29.010555 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8207d3-5ba4-473a-a624-39a68e4511bb None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:29.011131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8207d3-5ba4-473a-a624-39a68e4511bb None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:29.011131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8207d3-5ba4-473a-a624-39a68e4511bb None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:29.012511 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 719/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 05:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:06:29.017779 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a06ca-1693-4cdd-941f-056c6b90c0aa None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:29.018177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a06ca-1693-4cdd-941f-056c6b90c0aa None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:29.018335 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a06ca-1693-4cdd-941f-056c6b90c0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:29.086088 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-493a06ca-1693-4cdd-941f-056c6b90c0aa None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:06:29.087369 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-493a06ca-1693-4cdd-941f-056c6b90c0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:29.100067 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:29.183897 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 726/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 05:06:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:06:29.222388 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8207d3-5ba4-473a-a624-39a68e4511bb None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:06:29.223937 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-5b8207d3-5ba4-473a-a624-39a68e4511bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:29.327589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8207d3-5ba4-473a-a624-39a68e4511bb None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:06:29.340346 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:29.402016 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3ecde-7274-4cb7-8795-393f3e1926a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:06:29.402640 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3ecde-7274-4cb7-8795-393f3e1926a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:29.402959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3ecde-7274-4cb7-8795-393f3e1926a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:29.403244 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3ecde-7274-4cb7-8795-393f3e1926a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:29.404881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-91e3ecde-7274-4cb7-8795-393f3e1926a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:29.514818 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e3ecde-7274-4cb7-8795-393f3e1926a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:06:29.516286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e3ecde-7274-4cb7-8795-393f3e1926a6 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:06:29.534144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 720/1446] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:06:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:06:31.984423 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29167d57-676b-4b00-b6ba-f3468265bdbb None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:31.984955 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29167d57-676b-4b00-b6ba-f3468265bdbb None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:31.984955 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29167d57-676b-4b00-b6ba-f3468265bdbb None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:31.986237 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 727/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 05:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:06:31.992577 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecc870-437a-4ee4-886a-cceea8d84e6f None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:31.993026 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecc870-437a-4ee4-886a-cceea8d84e6f None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:31.993123 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecc870-437a-4ee4-886a-cceea8d84e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:32.042684 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:32.061354 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-daecc870-437a-4ee4-886a-cceea8d84e6f None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:06:32.063525 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-daecc870-437a-4ee4-886a-cceea8d84e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:32.165434 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 721/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 05:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:06:32.204525 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29167d57-676b-4b00-b6ba-f3468265bdbb None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:06:32.206085 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-29167d57-676b-4b00-b6ba-f3468265bdbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:32.326543 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29167d57-676b-4b00-b6ba-f3468265bdbb None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:06:32.369466 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:32.405062 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11344cf6-3718-4922-ad92-5f644a1024da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:06:32.405696 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11344cf6-3718-4922-ad92-5f644a1024da service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:32.405979 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11344cf6-3718-4922-ad92-5f644a1024da service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:32.406250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11344cf6-3718-4922-ad92-5f644a1024da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:32.407975 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-11344cf6-3718-4922-ad92-5f644a1024da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:32.520954 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11344cf6-3718-4922-ad92-5f644a1024da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:06:32.522413 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11344cf6-3718-4922-ad92-5f644a1024da service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:06:32.541393 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 728/1449] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:06:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:06:34.693634 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760ab70-8280-43fa-a03e-c2905eb78a88 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:34.694170 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760ab70-8280-43fa-a03e-c2905eb78a88 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:34.694170 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760ab70-8280-43fa-a03e-c2905eb78a88 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:34.695597 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 722/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 05:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:06:34.700945 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714276b4-0709-4c64-9134-99ac8ed1bf79 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:34.701388 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714276b4-0709-4c64-9134-99ac8ed1bf79 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:34.701581 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714276b4-0709-4c64-9134-99ac8ed1bf79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:34.769416 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-714276b4-0709-4c64-9134-99ac8ed1bf79 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:06:34.770633 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-714276b4-0709-4c64-9134-99ac8ed1bf79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:34.790960 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:34.993569 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 729/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 05:06:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:06:35.032680 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760ab70-8280-43fa-a03e-c2905eb78a88 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:06:35.034202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6760ab70-8280-43fa-a03e-c2905eb78a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:35.136361 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760ab70-8280-43fa-a03e-c2905eb78a88 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:06:35.155611 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:35.210115 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8689eeb8-d1b5-4b64-8af7-51cd8db9ab19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:06:35.210727 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8689eeb8-d1b5-4b64-8af7-51cd8db9ab19 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:35.211040 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8689eeb8-d1b5-4b64-8af7-51cd8db9ab19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:35.211316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8689eeb8-d1b5-4b64-8af7-51cd8db9ab19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:35.212922 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8689eeb8-d1b5-4b64-8af7-51cd8db9ab19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:35.335544 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8689eeb8-d1b5-4b64-8af7-51cd8db9ab19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:06:35.336903 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8689eeb8-d1b5-4b64-8af7-51cd8db9ab19 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:06:35.355463 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 723/1452] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:06:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:06:37.501698 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c627c44-3e06-479e-96d3-6bdb232473fd None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:37.502283 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c627c44-3e06-479e-96d3-6bdb232473fd None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:37.502283 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c627c44-3e06-479e-96d3-6bdb232473fd None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:37.504207 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 730/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 05:06:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:06:37.509639 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee568da-1031-4aff-a068-06f8bdabb40d None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:37.514031 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee568da-1031-4aff-a068-06f8bdabb40d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:37.514031 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee568da-1031-4aff-a068-06f8bdabb40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:37.575341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:37.578285 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-4ee568da-1031-4aff-a068-06f8bdabb40d None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:06:37.579596 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-4ee568da-1031-4aff-a068-06f8bdabb40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:37.683875 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 724/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 05:06:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:06:37.722010 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c627c44-3e06-479e-96d3-6bdb232473fd None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:06:37.723587 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-8c627c44-3e06-479e-96d3-6bdb232473fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:37.826036 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c627c44-3e06-479e-96d3-6bdb232473fd None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:06:37.875212 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:37.900369 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed4643-475e-42b4-9ca0-6603c24b91eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:06:37.900980 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed4643-475e-42b4-9ca0-6603c24b91eb service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:37.901255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed4643-475e-42b4-9ca0-6603c24b91eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:37.901533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed4643-475e-42b4-9ca0-6603c24b91eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:37.903286 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-0aed4643-475e-42b4-9ca0-6603c24b91eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:38.010992 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed4643-475e-42b4-9ca0-6603c24b91eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:06:38.012427 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed4643-475e-42b4-9ca0-6603c24b91eb service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:06:38.030144 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 731/1455] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:06:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:06:40.064600 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14674b-07a3-4b1e-9480-1317d2e46867 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:40.065136 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14674b-07a3-4b1e-9480-1317d2e46867 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:40.065136 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14674b-07a3-4b1e-9480-1317d2e46867 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:40.067127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 725/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 05:06:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 05:06:40.072528 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27233df0-199a-4633-8415-f53bffef9f72 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:40.072966 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27233df0-199a-4633-8415-f53bffef9f72 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:40.073130 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27233df0-199a-4633-8415-f53bffef9f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:40.138320 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-27233df0-199a-4633-8415-f53bffef9f72 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:06:40.139637 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-27233df0-199a-4633-8415-f53bffef9f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:40.168515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:40.241136 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 732/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 05:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:06:40.279211 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14674b-07a3-4b1e-9480-1317d2e46867 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:06:40.280754 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8a14674b-07a3-4b1e-9480-1317d2e46867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:40.399099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14674b-07a3-4b1e-9480-1317d2e46867 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:06:40.437050 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:40.481858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2cc2f6-60dd-49ea-be79-d56c29a81f4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:06:40.482525 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2cc2f6-60dd-49ea-be79-d56c29a81f4d service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:40.482792 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2cc2f6-60dd-49ea-be79-d56c29a81f4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:40.483098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2cc2f6-60dd-49ea-be79-d56c29a81f4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:40.484725 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2d2cc2f6-60dd-49ea-be79-d56c29a81f4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:40.600257 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2cc2f6-60dd-49ea-be79-d56c29a81f4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:06:40.601679 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2cc2f6-60dd-49ea-be79-d56c29a81f4d service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:06:40.621292 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 726/1458] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:06:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:06:42.723038 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a692ab-67d7-4392-858c-98beb6fa0d06 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:42.723578 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a692ab-67d7-4392-858c-98beb6fa0d06 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:42.723691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a692ab-67d7-4392-858c-98beb6fa0d06 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:42.724946 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 733/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 05:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:06:42.730513 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347f458-9254-4720-9458-4fe3907688a1 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:42.730978 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347f458-9254-4720-9458-4fe3907688a1 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:42.731078 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347f458-9254-4720-9458-4fe3907688a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:42.797015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-c347f458-9254-4720-9458-4fe3907688a1 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:06:42.798235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c347f458-9254-4720-9458-4fe3907688a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:42.806542 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:42.898412 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 727/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 05:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:06:42.935536 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a692ab-67d7-4392-858c-98beb6fa0d06 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:06:42.937080 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-85a692ab-67d7-4392-858c-98beb6fa0d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:43.038989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a692ab-67d7-4392-858c-98beb6fa0d06 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:06:43.093762 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:43.111774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6c130-89c2-4090-aeda-62c63f935e9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:06:43.112877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6c130-89c2-4090-aeda-62c63f935e9d service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:43.113159 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6c130-89c2-4090-aeda-62c63f935e9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:43.113435 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6c130-89c2-4090-aeda-62c63f935e9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:43.115090 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-edc6c130-89c2-4090-aeda-62c63f935e9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:43.221647 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6c130-89c2-4090-aeda-62c63f935e9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:06:43.223059 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6c130-89c2-4090-aeda-62c63f935e9d service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:06:43.242156 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 734/1461] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:06:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:06:45.286418 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a9b1e-c2f4-477d-b3f1-30e6d5d2400e None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:45.287015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a9b1e-c2f4-477d-b3f1-30e6d5d2400e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:45.287015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a9b1e-c2f4-477d-b3f1-30e6d5d2400e None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:45.288377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 728/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 05:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:06:45.293578 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bd6c2-1d29-4d97-8ac3-511222f6d98d None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:45.293966 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bd6c2-1d29-4d97-8ac3-511222f6d98d None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:45.294126 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bd6c2-1d29-4d97-8ac3-511222f6d98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:45.359216 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-5c7bd6c2-1d29-4d97-8ac3-511222f6d98d None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:06:45.360419 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-5c7bd6c2-1d29-4d97-8ac3-511222f6d98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:45.399042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:45.457722 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 735/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 05:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:06:45.495584 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a9b1e-c2f4-477d-b3f1-30e6d5d2400e None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:06:45.497138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-429a9b1e-c2f4-477d-b3f1-30e6d5d2400e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:45.600037 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a9b1e-c2f4-477d-b3f1-30e6d5d2400e None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:06:45.637627 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:45.682229 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c8637-90a7-4387-9132-2b3b3c37857e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:06:45.682871 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c8637-90a7-4387-9132-2b3b3c37857e service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:45.683157 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c8637-90a7-4387-9132-2b3b3c37857e service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:45.683432 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c8637-90a7-4387-9132-2b3b3c37857e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:45.685043 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-884c8637-90a7-4387-9132-2b3b3c37857e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:45.793802 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c8637-90a7-4387-9132-2b3b3c37857e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:06:45.795276 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c8637-90a7-4387-9132-2b3b3c37857e service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:06:45.814073 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 729/1464] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:06:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:06:47.851799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0c57c-9921-4763-872c-83d172c5f112 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:47.852355 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0c57c-9921-4763-872c-83d172c5f112 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:47.852545 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0c57c-9921-4763-872c-83d172c5f112 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:47.853735 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 736/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 05:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:06:47.859175 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902fea64-6682-43d7-8250-2a13bd776184 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:47.859566 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902fea64-6682-43d7-8250-2a13bd776184 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:47.859735 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902fea64-6682-43d7-8250-2a13bd776184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:47.928723 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-902fea64-6682-43d7-8250-2a13bd776184 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:06:47.929953 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-902fea64-6682-43d7-8250-2a13bd776184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:47.948103 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:48.028135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 730/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 05:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:06:48.063846 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0c57c-9921-4763-872c-83d172c5f112 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:06:48.065957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-aec0c57c-9921-4763-872c-83d172c5f112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:48.168777 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0c57c-9921-4763-872c-83d172c5f112 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:06:48.234458 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:48.241790 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc39f2-51a2-415f-ad73-b165e05ba55d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:06:48.242406 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc39f2-51a2-415f-ad73-b165e05ba55d service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:48.242695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc39f2-51a2-415f-ad73-b165e05ba55d service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:48.245743 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc39f2-51a2-415f-ad73-b165e05ba55d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:48.245743 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-b7cc39f2-51a2-415f-ad73-b165e05ba55d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:48.359800 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cc39f2-51a2-415f-ad73-b165e05ba55d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:06:48.361236 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cc39f2-51a2-415f-ad73-b165e05ba55d service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:06:48.379462 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 737/1467] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:06:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:06:50.394217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419c694-4fbd-4ae9-9799-4fad33fdb308 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:50.394868 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419c694-4fbd-4ae9-9799-4fad33fdb308 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:50.394868 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419c694-4fbd-4ae9-9799-4fad33fdb308 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:50.396279 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 731/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 05:06:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:06:50.401746 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fffd4-1606-43b1-88c7-16a40dddd5d0 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:50.402227 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fffd4-1606-43b1-88c7-16a40dddd5d0 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:50.402330 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fffd4-1606-43b1-88c7-16a40dddd5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:50.468456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-e89fffd4-1606-43b1-88c7-16a40dddd5d0 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:06:50.469681 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-e89fffd4-1606-43b1-88c7-16a40dddd5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:50.532709 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:50.583816 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 738/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 05:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:06:50.620611 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2419c694-4fbd-4ae9-9799-4fad33fdb308 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:06:50.622137 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2419c694-4fbd-4ae9-9799-4fad33fdb308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:50.728720 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2419c694-4fbd-4ae9-9799-4fad33fdb308 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:06:50.779862 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:06:50.805642 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff94eaa-f031-4d2f-b3f4-a1798ae25fe6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:06:50.806255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff94eaa-f031-4d2f-b3f4-a1798ae25fe6 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:06:50.806535 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff94eaa-f031-4d2f-b3f4-a1798ae25fe6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:06:50.806807 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff94eaa-f031-4d2f-b3f4-a1798ae25fe6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:06:50.809133 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-aff94eaa-f031-4d2f-b3f4-a1798ae25fe6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:06:50.919195 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff94eaa-f031-4d2f-b3f4-a1798ae25fe6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:06:50.920536 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff94eaa-f031-4d2f-b3f4-a1798ae25fe6 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:06:50.939077 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 732/1470] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:06:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:07:00.581885 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=22 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:07:00.936849 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=66 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:07:10.685160 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63476f45-ff3d-4588-9bed-70e213e95cd8 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:07:10.685666 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63476f45-ff3d-4588-9bed-70e213e95cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:07:10.685773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63476f45-ff3d-4588-9bed-70e213e95cd8 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:07:10.687119 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 739/1471] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:07:10.692618 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ecd5fb-2787-4ffb-9b10-11c9b7894508 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:07:10.693057 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ecd5fb-2787-4ffb-9b10-11c9b7894508 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:07:10.693169 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ecd5fb-2787-4ffb-9b10-11c9b7894508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:07:10.760786 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-d4ecd5fb-2787-4ffb-9b10-11c9b7894508 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:07:10.762001 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d4ecd5fb-2787-4ffb-9b10-11c9b7894508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:07:10.857821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 733/1472] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:07:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:07:10.944767 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f6b48-8bba-415a-99d0-59ff4b651994 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:07:10.944767 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f6b48-8bba-415a-99d0-59ff4b651994 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:07:10.944767 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f6b48-8bba-415a-99d0-59ff4b651994 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:07:10.947154 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 740/1473] 10.222.0.40 () {54 vars in 966 bytes} [Wed Aug 3 05:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:07:10.950825 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ba891-0dac-4539-ba63-9686c4fd4b61 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:07:10.951146 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ba891-0dac-4539-ba63-9686c4fd4b61 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:07:10.951421 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ba891-0dac-4539-ba63-9686c4fd4b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:07:11.027934 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-df5ba891-0dac-4539-ba63-9686c4fd4b61 None None] MFA Rules not processed for user `84cc14b33895443b893fab487bfe1858`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:07:11.029149 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-df5ba891-0dac-4539-ba63-9686c4fd4b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:07:11.049917 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:07:11.105783 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 734/1474] 10.222.0.40 () {58 vars in 1063 bytes} [Wed Aug 3 05:07:10 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:07:11.113424 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f6b48-8bba-415a-99d0-59ff4b651994 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:07:11.114992 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-880f6b48-8bba-415a-99d0-59ff4b651994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:07:11.218531 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f6b48-8bba-415a-99d0-59ff4b651994 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:07:11.297084 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64812053-93d1-45e9-a71c-841e9135928e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cc14b33895443b893fab487bfe1858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:07:11.297733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64812053-93d1-45e9-a71c-841e9135928e service glance] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:07:11.298018 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64812053-93d1-45e9-a71c-841e9135928e service glance] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:07:11.298294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64812053-93d1-45e9-a71c-841e9135928e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:07:11.300100 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-64812053-93d1-45e9-a71c-841e9135928e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:07:11.356598 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:07:11.411904 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64812053-93d1-45e9-a71c-841e9135928e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:07:11.413375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64812053-93d1-45e9-a71c-841e9135928e service glance] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:07:11.431563 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 741/1475] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:07:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:07:13.533139 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3455aa-cb7d-4390-9611-7b6a39d82968 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:07:13.533665 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3455aa-cb7d-4390-9611-7b6a39d82968 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:07:13.533665 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3455aa-cb7d-4390-9611-7b6a39d82968 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:07:13.535002 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 735/1476] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:07:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:07:13.540227 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63594de4-941f-4154-800f-13b2c53d40d2 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:07:13.540633 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63594de4-941f-4154-800f-13b2c53d40d2 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:07:13.540776 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63594de4-941f-4154-800f-13b2c53d40d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:07:13.607012 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-63594de4-941f-4154-800f-13b2c53d40d2 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:07:13.608233 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-63594de4-941f-4154-800f-13b2c53d40d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:07:13.715955 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 742/1477] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:07:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:07:13.739265 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3455aa-cb7d-4390-9611-7b6a39d82968 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:07:13.741104 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-cc3455aa-cb7d-4390-9611-7b6a39d82968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:07:13.865559 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3455aa-cb7d-4390-9611-7b6a39d82968 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:07:13.913943 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:07:13.960396 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b310c-8aee-409b-a9f3-e7b22edc22cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:07:13.961018 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b310c-8aee-409b-a9f3-e7b22edc22cf admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:07:13.961300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b310c-8aee-409b-a9f3-e7b22edc22cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:07:13.961578 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b310c-8aee-409b-a9f3-e7b22edc22cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:07:13.962270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b310c-8aee-409b-a9f3-e7b22edc22cf admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:07:13.964027 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: WARNING py.warnings [None req-965b310c-8aee-409b-a9f3-e7b22edc22cf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 05:07:13.964027 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: warnings.warn(msg) Aug 03 05:07:13.964027 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: Aug 03 05:07:13.965181 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b310c-8aee-409b-a9f3-e7b22edc22cf admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:07:13.983648 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 736/1478] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Aug 3 05:07:13 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:07:16.003601 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7252218-5d75-41bc-bca8-48f3ddbc39e6 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:07:16.004160 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7252218-5d75-41bc-bca8-48f3ddbc39e6 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:07:16.004160 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7252218-5d75-41bc-bca8-48f3ddbc39e6 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:07:16.005486 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 743/1479] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:07:16.010747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca6323-0db5-4d40-8a15-fa2e8ceec9d0 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:07:16.011257 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca6323-0db5-4d40-8a15-fa2e8ceec9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:07:16.011420 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca6323-0db5-4d40-8a15-fa2e8ceec9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:07:16.076813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-1cca6323-0db5-4d40-8a15-fa2e8ceec9d0 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:07:16.078017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1cca6323-0db5-4d40-8a15-fa2e8ceec9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:07:16.174290 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 737/1480] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:07:16.217721 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7252218-5d75-41bc-bca8-48f3ddbc39e6 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:07:16.219299 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-d7252218-5d75-41bc-bca8-48f3ddbc39e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:07:16.229640 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:07:16.367120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7252218-5d75-41bc-bca8-48f3ddbc39e6 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:07:16.446691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:07:16.479306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f348734c-48a6-472a-b613-1e9226e6c005 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:07:16.479854 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348734c-48a6-472a-b613-1e9226e6c005 service nova] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:07:16.480134 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348734c-48a6-472a-b613-1e9226e6c005 service nova] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:07:16.480407 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348734c-48a6-472a-b613-1e9226e6c005 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:07:16.482016 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-f348734c-48a6-472a-b613-1e9226e6c005 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:07:16.570817 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f348734c-48a6-472a-b613-1e9226e6c005 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:07:16.572472 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f348734c-48a6-472a-b613-1e9226e6c005 service nova] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:07:16.590591 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 744/1481] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:07:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:07:18.787878 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137a48e-ca40-40eb-b9c9-e5347ac0865e None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:07:18.788408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137a48e-ca40-40eb-b9c9-e5347ac0865e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:07:18.788564 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137a48e-ca40-40eb-b9c9-e5347ac0865e None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:07:18.789855 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 738/1482] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:07:18.795550 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122221d-5490-4122-91ba-fc35ee07f924 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:07:18.796004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122221d-5490-4122-91ba-fc35ee07f924 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:07:18.796176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122221d-5490-4122-91ba-fc35ee07f924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:07:18.864412 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-a122221d-5490-4122-91ba-fc35ee07f924 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:07:18.865901 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-a122221d-5490-4122-91ba-fc35ee07f924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:07:18.913175 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:07:18.964919 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 745/1483] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:07:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:07:19.002150 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5137a48e-ca40-40eb-b9c9-e5347ac0865e None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:07:19.003701 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-5137a48e-ca40-40eb-b9c9-e5347ac0865e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:07:19.026985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:07:19.105702 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5137a48e-ca40-40eb-b9c9-e5347ac0865e None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:07:19.179168 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52591c4d-0ad4-4ad0-90a3-a5695ffee944 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:07:19.179785 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52591c4d-0ad4-4ad0-90a3-a5695ffee944 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:07:19.180072 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52591c4d-0ad4-4ad0-90a3-a5695ffee944 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:07:19.180346 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52591c4d-0ad4-4ad0-90a3-a5695ffee944 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:07:19.181969 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-52591c4d-0ad4-4ad0-90a3-a5695ffee944 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:07:19.255852 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:07:19.291867 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52591c4d-0ad4-4ad0-90a3-a5695ffee944 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:07:19.293314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52591c4d-0ad4-4ad0-90a3-a5695ffee944 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:07:19.311017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 739/1484] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:07:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:07:21.375101 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9f69d-33bc-4ce7-8807-5eee870577c3 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:07:21.375756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9f69d-33bc-4ce7-8807-5eee870577c3 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:07:21.375756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9f69d-33bc-4ce7-8807-5eee870577c3 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:07:21.377025 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 746/1485] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:07:21.382304 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26defcfd-2d2b-4d17-94df-7499d8ce63dd None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:07:21.382767 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26defcfd-2d2b-4d17-94df-7499d8ce63dd None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:07:21.383202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26defcfd-2d2b-4d17-94df-7499d8ce63dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:07:21.450110 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-26defcfd-2d2b-4d17-94df-7499d8ce63dd None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:07:21.451373 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-26defcfd-2d2b-4d17-94df-7499d8ce63dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:07:21.548811 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 740/1486] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:07:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:07:21.587716 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9f69d-33bc-4ce7-8807-5eee870577c3 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:07:21.589250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-1ed9f69d-33bc-4ce7-8807-5eee870577c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:07:21.693329 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9f69d-33bc-4ce7-8807-5eee870577c3 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:07:21.764840 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:07:21.767381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b007658-b1f5-4802-ab3a-aa8b464cec7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:07:21.768012 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b007658-b1f5-4802-ab3a-aa8b464cec7f service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:07:21.768298 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b007658-b1f5-4802-ab3a-aa8b464cec7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:07:21.768577 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b007658-b1f5-4802-ab3a-aa8b464cec7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:07:21.771878 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-5b007658-b1f5-4802-ab3a-aa8b464cec7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:07:21.888260 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b007658-b1f5-4802-ab3a-aa8b464cec7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:07:21.889700 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b007658-b1f5-4802-ab3a-aa8b464cec7f service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:07:21.909262 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 747/1487] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:07:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:07:24.029506 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f068a-1e95-4a48-856e-b9bccc1676d5 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:07:24.030080 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f068a-1e95-4a48-856e-b9bccc1676d5 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:07:24.030080 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f068a-1e95-4a48-856e-b9bccc1676d5 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:07:24.031486 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 741/1488] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:07:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:07:24.038978 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c57fa1-ba20-41f2-9c24-886d5e52e178 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:07:24.039646 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c57fa1-ba20-41f2-9c24-886d5e52e178 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:07:24.039646 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c57fa1-ba20-41f2-9c24-886d5e52e178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:07:24.106312 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-e8c57fa1-ba20-41f2-9c24-886d5e52e178 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:07:24.107549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-e8c57fa1-ba20-41f2-9c24-886d5e52e178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:07:24.159467 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:07:24.203280 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 748/1489] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:07:24.243707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f068a-1e95-4a48-856e-b9bccc1676d5 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:07:24.245270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-579f068a-1e95-4a48-856e-b9bccc1676d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:07:24.282956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:07:24.352375 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f068a-1e95-4a48-856e-b9bccc1676d5 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:07:24.429340 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef7deb-0e7e-41a7-967e-1670fe7ee4d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:07:24.429964 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef7deb-0e7e-41a7-967e-1670fe7ee4d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:07:24.430245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef7deb-0e7e-41a7-967e-1670fe7ee4d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:07:24.430517 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef7deb-0e7e-41a7-967e-1670fe7ee4d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:07:24.432180 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-89ef7deb-0e7e-41a7-967e-1670fe7ee4d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:07:24.511981 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:07:24.540035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ef7deb-0e7e-41a7-967e-1670fe7ee4d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:07:24.541456 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ef7deb-0e7e-41a7-967e-1670fe7ee4d2 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:07:24.559852 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 742/1490] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:07:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:07:26.903104 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe67057-25ee-4452-82cf-0b457c3d914f None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:07:26.903667 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe67057-25ee-4452-82cf-0b457c3d914f None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:07:26.903667 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe67057-25ee-4452-82cf-0b457c3d914f None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:07:26.905017 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 749/1491] 10.222.0.40 () {54 vars in 926 bytes} [Wed Aug 3 05:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:07:26.910343 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e8760-1345-4fcd-82b2-e47edc426c44 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:07:26.910759 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e8760-1345-4fcd-82b2-e47edc426c44 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:07:26.910947 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e8760-1345-4fcd-82b2-e47edc426c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:07:26.977248 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-f41e8760-1345-4fcd-82b2-e47edc426c44 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:07:26.978395 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f41e8760-1345-4fcd-82b2-e47edc426c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:07:27.083632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 743/1492] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Aug 3 05:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:07:27.123458 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe67057-25ee-4452-82cf-0b457c3d914f None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:07:27.125489 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-0fe67057-25ee-4452-82cf-0b457c3d914f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:07:27.228540 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe67057-25ee-4452-82cf-0b457c3d914f None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:07:27.301090 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572afe17-38c5-4c3a-adbb-64e5f9fe390a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:07:27.301711 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572afe17-38c5-4c3a-adbb-64e5f9fe390a service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:07:27.301995 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572afe17-38c5-4c3a-adbb-64e5f9fe390a service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:07:27.302271 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572afe17-38c5-4c3a-adbb-64e5f9fe390a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:07:27.303963 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-572afe17-38c5-4c3a-adbb-64e5f9fe390a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:07:27.313625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:07:27.414463 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572afe17-38c5-4c3a-adbb-64e5f9fe390a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:07:27.415960 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572afe17-38c5-4c3a-adbb-64e5f9fe390a service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:07:27.435608 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 750/1493] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:07:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:07:37.081741 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=82 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:07:37.440663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=50 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:09:02.242747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3565a7-3061-4c85-8330-09b31eb4b5d0 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:09:02.243311 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3565a7-3061-4c85-8330-09b31eb4b5d0 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:09:02.243419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3565a7-3061-4c85-8330-09b31eb4b5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:09:02.322125 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-4a3565a7-3061-4c85-8330-09b31eb4b5d0 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:09:02.323386 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-4a3565a7-3061-4c85-8330-09b31eb4b5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:09:02.419274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 744/1494] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:09:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:09:02.427555 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c042bd-5e3e-4ac0-afa6-ed770ad5bf68 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:09:02.428052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c042bd-5e3e-4ac0-afa6-ed770ad5bf68 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:09:02.428169 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c042bd-5e3e-4ac0-afa6-ed770ad5bf68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:09:02.506888 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-92c042bd-5e3e-4ac0-afa6-ed770ad5bf68 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:09:02.508030 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-92c042bd-5e3e-4ac0-afa6-ed770ad5bf68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:09:02.602361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 751/1495] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:09:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:09:02.607217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3565a7-3061-4c85-8330-09b31eb4b5d0 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:09:02.608498 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-4a3565a7-3061-4c85-8330-09b31eb4b5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:09:02.620302 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3565a7-3061-4c85-8330-09b31eb4b5d0 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:09:02.683181 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:09:02.717435 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ea80a-0304-42bd-ab21-af5c1661133b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:09:02.718053 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ea80a-0304-42bd-ab21-af5c1661133b admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:09:02.718337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ea80a-0304-42bd-ab21-af5c1661133b admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:09:02.718613 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ea80a-0304-42bd-ab21-af5c1661133b admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:09:02.719348 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ea80a-0304-42bd-ab21-af5c1661133b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:09:02.719654 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ea80a-0304-42bd-ab21-af5c1661133b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:09:02.720673 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: WARNING py.warnings [None req-472ea80a-0304-42bd-ab21-af5c1661133b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 03 05:09:02.720673 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: warnings.warn(msg) Aug 03 05:09:02.720673 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: Aug 03 05:09:02.721566 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ea80a-0304-42bd-ab21-af5c1661133b admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:09:02.732996 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 745/1496] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:09:02.741572 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c042bd-5e3e-4ac0-afa6-ed770ad5bf68 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:09:02.751235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c042bd-5e3e-4ac0-afa6-ed770ad5bf68 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:09:02.812496 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:09:02.845486 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59562b-216c-45c3-b190-0381b4cc76cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:09:02.846097 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59562b-216c-45c3-b190-0381b4cc76cb admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:09:02.846377 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59562b-216c-45c3-b190-0381b4cc76cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:09:02.846649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59562b-216c-45c3-b190-0381b4cc76cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:09:02.847331 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf59562b-216c-45c3-b190-0381b4cc76cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:09:02.849068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: WARNING py.warnings [None req-cf59562b-216c-45c3-b190-0381b4cc76cb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 05:09:02.849068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: warnings.warn(msg) Aug 03 05:09:02.849068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: Aug 03 05:09:02.850271 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf59562b-216c-45c3-b190-0381b4cc76cb admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:09:02.953865 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 752/1497] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:09:02 2022] POST /identity/v3/projects => generated 379 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:09:02.958056 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ea80a-0304-42bd-ab21-af5c1661133b admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:09:02.959610 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-472ea80a-0304-42bd-ab21-af5c1661133b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:09:02.997385 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ea80a-0304-42bd-ab21-af5c1661133b admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:09:03.097678 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc2e43-c007-44dc-831f-0b9998671b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:09:03.098208 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc2e43-c007-44dc-831f-0b9998671b87 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:09:03.098491 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc2e43-c007-44dc-831f-0b9998671b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:09:03.098766 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc2e43-c007-44dc-831f-0b9998671b87 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:09:03.099521 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc2e43-c007-44dc-831f-0b9998671b87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:09:03.100681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: WARNING py.warnings [None req-86cc2e43-c007-44dc-831f-0b9998671b87 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 05:09:03.100681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: warnings.warn(msg) Aug 03 05:09:03.100681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: Aug 03 05:09:03.101907 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc2e43-c007-44dc-831f-0b9998671b87 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:09:03.203946 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 746/1498] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:09:02 2022] POST /identity/v3/users => generated 290 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:09:03.208136 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59562b-216c-45c3-b190-0381b4cc76cb admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:09:03.209673 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-cf59562b-216c-45c3-b190-0381b4cc76cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:09:03.272004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59562b-216c-45c3-b190-0381b4cc76cb admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:09:03.357014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:09:03.368503 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17973f03-ac76-48e1-aba3-26a18a10845b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:09:03.369126 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17973f03-ac76-48e1-aba3-26a18a10845b admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:09:03.369409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17973f03-ac76-48e1-aba3-26a18a10845b admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:09:03.369685 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17973f03-ac76-48e1-aba3-26a18a10845b admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:09:03.370377 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17973f03-ac76-48e1-aba3-26a18a10845b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:09:03.372004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: WARNING py.warnings [None req-17973f03-ac76-48e1-aba3-26a18a10845b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 03 05:09:03.372004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: warnings.warn(msg) Aug 03 05:09:03.372004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: Aug 03 05:09:03.372890 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17973f03-ac76-48e1-aba3-26a18a10845b admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:09:03.385107 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 753/1499] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:09:03.389898 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc2e43-c007-44dc-831f-0b9998671b87 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:09:03.391579 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-86cc2e43-c007-44dc-831f-0b9998671b87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:09:03.402767 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc2e43-c007-44dc-831f-0b9998671b87 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:09:03.427612 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:09:03.500946 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877ed63-b8f1-42ab-9588-3d589261bc2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:09:03.501574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877ed63-b8f1-42ab-9588-3d589261bc2d admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:09:03.501857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877ed63-b8f1-42ab-9588-3d589261bc2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:09:03.502135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877ed63-b8f1-42ab-9588-3d589261bc2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:09:03.502832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877ed63-b8f1-42ab-9588-3d589261bc2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:09:03.503884 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: WARNING py.warnings [None req-d877ed63-b8f1-42ab-9588-3d589261bc2d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 03 05:09:03.503884 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: warnings.warn(msg) Aug 03 05:09:03.503884 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: Aug 03 05:09:03.505043 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877ed63-b8f1-42ab-9588-3d589261bc2d admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:09:03.518985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 747/1500] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:09:03.523039 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17973f03-ac76-48e1-aba3-26a18a10845b admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:09:03.524568 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-17973f03-ac76-48e1-aba3-26a18a10845b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:09:03.561489 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17973f03-ac76-48e1-aba3-26a18a10845b admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:09:03.665109 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ca222-d981-4edc-aeb7-0130d3c2862e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:09:03.665573 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ca222-d981-4edc-aeb7-0130d3c2862e admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:09:03.665868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ca222-d981-4edc-aeb7-0130d3c2862e admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:09:03.666147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ca222-d981-4edc-aeb7-0130d3c2862e admin admin] PATH_INFO: `/v3/projects/52c0354b6a3b431bba0808103da4b4ba/users/b9de311c8a2d4d829e51368483e6bdfe/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:09:03.690814 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ca222-d981-4edc-aeb7-0130d3c2862e admin admin] RBAC: Authorizing `identity:create_grant(project_id=52c0354b6a3b431bba0808103da4b4ba, user_id=b9de311c8a2d4d829e51368483e6bdfe, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:09:03.691893 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: WARNING py.warnings [None req-a00ca222-d981-4edc-aeb7-0130d3c2862e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 05:09:03.691893 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: warnings.warn(msg) Aug 03 05:09:03.691893 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: Aug 03 05:09:03.694119 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ca222-d981-4edc-aeb7-0130d3c2862e admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:09:03.816244 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 754/1501] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:09:03 2022] PUT /identity/v3/projects/52c0354b6a3b431bba0808103da4b4ba/users/b9de311c8a2d4d829e51368483e6bdfe/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:09:03.828953 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1eadf-7e90-400b-bf9b-9e4fe06336a3 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:09:03.829344 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1eadf-7e90-400b-bf9b-9e4fe06336a3 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:09:03.829631 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1eadf-7e90-400b-bf9b-9e4fe06336a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:09:03.887951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-72e1eadf-7e90-400b-bf9b-9e4fe06336a3 None None] MFA Rules not processed for user `b9de311c8a2d4d829e51368483e6bdfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:09:03.889170 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-72e1eadf-7e90-400b-bf9b-9e4fe06336a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:09:03.969927 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 748/1502] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:09:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:09:03.984691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4870c-d7dd-4909-a91f-e0a81f447971 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:09:03.984975 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4870c-d7dd-4909-a91f-e0a81f447971 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:09:03.985905 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4870c-d7dd-4909-a91f-e0a81f447971 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:09:03.986833 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 755/1503] 10.222.0.40 () {54 vars in 964 bytes} [Wed Aug 3 05:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:09:03.991772 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ca55d-3466-4f31-b926-5ecd3b1c3d73 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:09:03.992250 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ca55d-3466-4f31-b926-5ecd3b1c3d73 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:09:03.992541 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ca55d-3466-4f31-b926-5ecd3b1c3d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:09:04.024779 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:09:04.106548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-fb2ca55d-3466-4f31-b926-5ecd3b1c3d73 None None] MFA Rules not processed for user `c81ca55d52244bf1bfe4e829b6880c20`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:09:04.107793 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-fb2ca55d-3466-4f31-b926-5ecd3b1c3d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:09:04.182252 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 749/1504] 10.222.0.40 () {58 vars in 1061 bytes} [Wed Aug 3 05:09:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:09:04.189925 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4870c-d7dd-4909-a91f-e0a81f447971 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:09:04.191464 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-50c4870c-d7dd-4909-a91f-e0a81f447971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:09:04.241684 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:09:04.296183 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4870c-d7dd-4909-a91f-e0a81f447971 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:09:04.369634 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f7649-5b11-4d24-be60-bb8852c2c1ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:09:04.370249 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f7649-5b11-4d24-be60-bb8852c2c1ad service cinder] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:09:04.370526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f7649-5b11-4d24-be60-bb8852c2c1ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:09:04.370796 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f7649-5b11-4d24-be60-bb8852c2c1ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:09:04.372444 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-da5f7649-5b11-4d24-be60-bb8852c2c1ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:09:04.469711 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f7649-5b11-4d24-be60-bb8852c2c1ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:09:04.471204 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f7649-5b11-4d24-be60-bb8852c2c1ad service cinder] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:09:04.474660 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:09:04.490168 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 756/1505] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:09:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:09:04.536058 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966afce-fff6-4852-bcde-b71b675d10ab None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:09:04.536488 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966afce-fff6-4852-bcde-b71b675d10ab None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:09:04.536620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966afce-fff6-4852-bcde-b71b675d10ab None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:09:04.537895 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 750/1506] 10.222.0.40 () {52 vars in 830 bytes} [Wed Aug 3 05:09:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:09:04.541426 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f7649-5b11-4d24-be60-bb8852c2c1ad service cinder] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:09:04.542661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-da5f7649-5b11-4d24-be60-bb8852c2c1ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:09:04.642768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f7649-5b11-4d24-be60-bb8852c2c1ad service cinder] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:09:04.739161 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ad159-c2d5-4ca8-899e-1d00620a5e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:09:04.739803 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ad159-c2d5-4ca8-899e-1d00620a5e99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:09:04.740087 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ad159-c2d5-4ca8-899e-1d00620a5e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:09:04.740363 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ad159-c2d5-4ca8-899e-1d00620a5e99 admin admin] PATH_INFO: `/v3/users/b9de311c8a2d4d829e51368483e6bdfe` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:09:04.753392 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07ad159-c2d5-4ca8-899e-1d00620a5e99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9de311c8a2d4d829e51368483e6bdfe)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:09:04.754372 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: WARNING py.warnings [None req-e07ad159-c2d5-4ca8-899e-1d00620a5e99 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 05:09:04.754372 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: warnings.warn(msg) Aug 03 05:09:04.754372 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: Aug 03 05:09:04.755637 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07ad159-c2d5-4ca8-899e-1d00620a5e99 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:09:04.766137 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:09:04.956440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-e07ad159-c2d5-4ca8-899e-1d00620a5e99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9de311c8a2d4d829e51368483e6bdfe'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:09:05.002819 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-e07ad159-c2d5-4ca8-899e-1d00620a5e99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9de311c8a2d4d829e51368483e6bdfe'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:09:05.003388 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-e07ad159-c2d5-4ca8-899e-1d00620a5e99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9de311c8a2d4d829e51368483e6bdfe'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:09:05.027049 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-e07ad159-c2d5-4ca8-899e-1d00620a5e99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9de311c8a2d4d829e51368483e6bdfe'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:09:05.027523 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-e07ad159-c2d5-4ca8-899e-1d00620a5e99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9de311c8a2d4d829e51368483e6bdfe'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:09:05.047351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 757/1507] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:09:04 2022] DELETE /identity/v3/users/b9de311c8a2d4d829e51368483e6bdfe => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:09:05.051893 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06056d17-cafa-4ac7-a2d9-5c7679d4a3ee None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:09:05.052181 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06056d17-cafa-4ac7-a2d9-5c7679d4a3ee None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:09:05.052457 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06056d17-cafa-4ac7-a2d9-5c7679d4a3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:09:05.117860 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-06056d17-cafa-4ac7-a2d9-5c7679d4a3ee None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:09:05.119108 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-06056d17-cafa-4ac7-a2d9-5c7679d4a3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:09:05.193855 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:09:05.205409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 751/1508] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:09:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:09:05.215869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ad159-c2d5-4ca8-899e-1d00620a5e99 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:09:05.217091 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-e07ad159-c2d5-4ca8-899e-1d00620a5e99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:09:05.320734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ad159-c2d5-4ca8-899e-1d00620a5e99 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:09:05.374120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:09:05.393011 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dabd052-b49f-4140-862e-5e15a87f71e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:09:05.393835 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabd052-b49f-4140-862e-5e15a87f71e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:09:05.394125 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabd052-b49f-4140-862e-5e15a87f71e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:09:05.394417 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabd052-b49f-4140-862e-5e15a87f71e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:09:05.396121 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-9dabd052-b49f-4140-862e-5e15a87f71e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:09:05.504805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dabd052-b49f-4140-862e-5e15a87f71e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:09:05.506250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dabd052-b49f-4140-862e-5e15a87f71e3 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:09:05.524201 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 758/1509] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:09:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:09:05.999080 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06056d17-cafa-4ac7-a2d9-5c7679d4a3ee None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:09:05.999080 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-06056d17-cafa-4ac7-a2d9-5c7679d4a3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:09:06.007623 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06056d17-cafa-4ac7-a2d9-5c7679d4a3ee None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:09:06.115533 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd914dc-04b9-4199-8998-9b19118fbe64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:09:06.116184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd914dc-04b9-4199-8998-9b19118fbe64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:09:06.116470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd914dc-04b9-4199-8998-9b19118fbe64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:09:06.116746 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd914dc-04b9-4199-8998-9b19118fbe64 admin admin] PATH_INFO: `/v3/projects/52c0354b6a3b431bba0808103da4b4ba` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:09:06.127206 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd914dc-04b9-4199-8998-9b19118fbe64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52c0354b6a3b431bba0808103da4b4ba)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:09:06.128236 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: WARNING py.warnings [None req-3bd914dc-04b9-4199-8998-9b19118fbe64 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 05:09:06.128236 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: warnings.warn(msg) Aug 03 05:09:06.128236 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: Aug 03 05:09:06.129443 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd914dc-04b9-4199-8998-9b19118fbe64 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:09:06.236592 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-3bd914dc-04b9-4199-8998-9b19118fbe64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52c0354b6a3b431bba0808103da4b4ba', 'request_id': 'req-3bd914dc-04b9-4199-8998-9b19118fbe64'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:09:06.300043 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-3bd914dc-04b9-4199-8998-9b19118fbe64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52c0354b6a3b431bba0808103da4b4ba', 'request_id': 'req-3bd914dc-04b9-4199-8998-9b19118fbe64'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:09:06.307011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-3bd914dc-04b9-4199-8998-9b19118fbe64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52c0354b6a3b431bba0808103da4b4ba', 'request_id': 'req-3bd914dc-04b9-4199-8998-9b19118fbe64'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:09:06.336360 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-3bd914dc-04b9-4199-8998-9b19118fbe64 admin admin] The token cache is being invalidate because project 52c0354b6a3b431bba0808103da4b4ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:09:06.336644 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-3bd914dc-04b9-4199-8998-9b19118fbe64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:09:06.337170 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-3bd914dc-04b9-4199-8998-9b19118fbe64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:09:06.338503 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 752/1510] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:09:05 2022] DELETE /identity/v3/projects/52c0354b6a3b431bba0808103da4b4ba => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:09:15.521935 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=62 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:09:16.334245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:10:46.160214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8dd17e-f213-4a10-a043-7a11951fc235 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:10:46.160838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8dd17e-f213-4a10-a043-7a11951fc235 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:10:46.160838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8dd17e-f213-4a10-a043-7a11951fc235 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:10:46.162126 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 759/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 05:10:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:10:46.167755 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def50989-95e1-4a0a-bbd8-382cfcad40e9 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:10:46.168195 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def50989-95e1-4a0a-bbd8-382cfcad40e9 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:10:46.168330 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def50989-95e1-4a0a-bbd8-382cfcad40e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:10:46.250025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-def50989-95e1-4a0a-bbd8-382cfcad40e9 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:10:46.251258 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-def50989-95e1-4a0a-bbd8-382cfcad40e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:10:46.356338 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 753/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 05:10:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:10:46.508425 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8dd17e-f213-4a10-a043-7a11951fc235 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:10:46.510623 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-2f8dd17e-f213-4a10-a043-7a11951fc235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:10:46.632909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8dd17e-f213-4a10-a043-7a11951fc235 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:10:46.638957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe861c37-1a62-4ee6-9cf8-3296da171eeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:10:46.639592 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe861c37-1a62-4ee6-9cf8-3296da171eeb service nova] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:10:46.639882 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe861c37-1a62-4ee6-9cf8-3296da171eeb service nova] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:10:46.640166 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe861c37-1a62-4ee6-9cf8-3296da171eeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:10:46.648747 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe861c37-1a62-4ee6-9cf8-3296da171eeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:10:46.650973 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe861c37-1a62-4ee6-9cf8-3296da171eeb service nova] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:10:46.660965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 760/1513] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:10:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:10:47.500125 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75740f9-0fa8-4343-af4e-0b8c0dfa6f02 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:10:47.500703 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75740f9-0fa8-4343-af4e-0b8c0dfa6f02 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:10:47.500703 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75740f9-0fa8-4343-af4e-0b8c0dfa6f02 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:10:47.502115 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 754/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 05:10:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:10:47.506982 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526d419-d76c-44bc-9c4e-4244642bc0b5 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:10:47.511810 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526d419-d76c-44bc-9c4e-4244642bc0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:10:47.511810 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526d419-d76c-44bc-9c4e-4244642bc0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:10:47.528032 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-6526d419-d76c-44bc-9c4e-4244642bc0b5 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:10:47.529317 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-6526d419-d76c-44bc-9c4e-4244642bc0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:10:47.537206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 761/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 05:10:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:10:47.692452 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75740f9-0fa8-4343-af4e-0b8c0dfa6f02 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:10:47.707226 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75740f9-0fa8-4343-af4e-0b8c0dfa6f02 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:10:47.713164 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aef9e0-f4b1-4be4-aad6-7aed18a2d943 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:10:47.713482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aef9e0-f4b1-4be4-aad6-7aed18a2d943 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:10:47.713763 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aef9e0-f4b1-4be4-aad6-7aed18a2d943 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:10:47.714037 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aef9e0-f4b1-4be4-aad6-7aed18a2d943 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:10:47.722410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aef9e0-f4b1-4be4-aad6-7aed18a2d943 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:10:47.724113 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aef9e0-f4b1-4be4-aad6-7aed18a2d943 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:10:47.734578 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 755/1516] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:10:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:10:48.113817 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27f958-55af-4928-be70-5fe780db383c None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:10:48.114119 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27f958-55af-4928-be70-5fe780db383c None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:10:48.114397 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27f958-55af-4928-be70-5fe780db383c None None] PATH_INFO: `/v3` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:10:48.115717 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 762/1517] 10.222.0.67 () {54 vars in 925 bytes} [Wed Aug 3 05:10:48 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:10:48.123206 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91852483-8d0c-41d2-bfce-88286854517c None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:10:48.123534 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91852483-8d0c-41d2-bfce-88286854517c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:10:48.123816 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91852483-8d0c-41d2-bfce-88286854517c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:10:48.183500 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-91852483-8d0c-41d2-bfce-88286854517c None None] MFA Rules not processed for user `59846d88f4314c2382aa5cb8bae5053b`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:10:48.184718 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-91852483-8d0c-41d2-bfce-88286854517c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:10:48.201940 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:10:48.265025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 756/1518] 10.222.0.67 () {58 vars in 1015 bytes} [Wed Aug 3 05:10:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:10:52.262234 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731fc89c-03cd-4169-9b87-d3c214bd3bb8 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:10:52.262791 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731fc89c-03cd-4169-9b87-d3c214bd3bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:10:52.262943 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731fc89c-03cd-4169-9b87-d3c214bd3bb8 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:10:52.264184 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 763/1519] 10.222.0.40 () {54 vars in 961 bytes} [Wed Aug 3 05:10:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:10:52.269133 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd28810-e442-41b7-bbb0-8b453bd193aa None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:10:52.270005 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd28810-e442-41b7-bbb0-8b453bd193aa None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:10:52.270005 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd28810-e442-41b7-bbb0-8b453bd193aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:10:52.288611 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-afd28810-e442-41b7-bbb0-8b453bd193aa None None] MFA Rules not processed for user `59846d88f4314c2382aa5cb8bae5053b`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:10:52.289804 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-afd28810-e442-41b7-bbb0-8b453bd193aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:10:52.298265 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 757/1520] 10.222.0.40 () {58 vars in 1058 bytes} [Wed Aug 3 05:10:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:10:52.305956 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731fc89c-03cd-4169-9b87-d3c214bd3bb8 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:10:52.319794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731fc89c-03cd-4169-9b87-d3c214bd3bb8 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:10:52.325379 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17486463-c947-478e-9ee7-e91333b8818c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59846d88f4314c2382aa5cb8bae5053b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:10:52.325977 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17486463-c947-478e-9ee7-e91333b8818c service placement] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:10:52.326254 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17486463-c947-478e-9ee7-e91333b8818c service placement] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:10:52.326527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17486463-c947-478e-9ee7-e91333b8818c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:10:52.334937 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17486463-c947-478e-9ee7-e91333b8818c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:10:52.336532 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17486463-c947-478e-9ee7-e91333b8818c service placement] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:10:52.346659 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 764/1521] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:10:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:10:58.966915 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523fa53d-93d3-47e8-975a-36472e451b58 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:10:58.967491 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523fa53d-93d3-47e8-975a-36472e451b58 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:10:58.967491 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523fa53d-93d3-47e8-975a-36472e451b58 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:10:58.968877 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 758/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 05:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:10:58.974779 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c51ec9-a21e-42eb-b0ec-f10d1e9c3077 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:10:58.975290 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c51ec9-a21e-42eb-b0ec-f10d1e9c3077 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:10:58.975394 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c51ec9-a21e-42eb-b0ec-f10d1e9c3077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:10:58.995893 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-d3c51ec9-a21e-42eb-b0ec-f10d1e9c3077 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:10:58.997117 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-d3c51ec9-a21e-42eb-b0ec-f10d1e9c3077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:10:59.005311 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 765/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 05:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:10:59.159732 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523fa53d-93d3-47e8-975a-36472e451b58 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:10:59.174657 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523fa53d-93d3-47e8-975a-36472e451b58 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:10:59.180377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c92987-0439-4370-952f-fce9a646aafc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:10:59.181485 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c92987-0439-4370-952f-fce9a646aafc service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:10:59.181785 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c92987-0439-4370-952f-fce9a646aafc service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:10:59.182058 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c92987-0439-4370-952f-fce9a646aafc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:10:59.190369 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c92987-0439-4370-952f-fce9a646aafc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:10:59.195892 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c92987-0439-4370-952f-fce9a646aafc service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:10:59.201733 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 759/1524] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:10:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:11:00.024106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04688b-b02a-4f64-8200-aad9e223ee68 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:11:00.024811 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04688b-b02a-4f64-8200-aad9e223ee68 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:11:00.024811 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04688b-b02a-4f64-8200-aad9e223ee68 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:11:00.026020 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 766/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 05:11:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:11:00.031083 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941e4e6-2b16-45ee-9f1d-014a921c6241 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:11:00.031528 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941e4e6-2b16-45ee-9f1d-014a921c6241 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:11:00.031637 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941e4e6-2b16-45ee-9f1d-014a921c6241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:11:00.050933 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-2941e4e6-2b16-45ee-9f1d-014a921c6241 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:11:00.065080 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2941e4e6-2b16-45ee-9f1d-014a921c6241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:11:00.065080 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 760/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 05:11:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:11:00.215249 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04688b-b02a-4f64-8200-aad9e223ee68 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:11:00.229609 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04688b-b02a-4f64-8200-aad9e223ee68 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:11:00.235342 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d82b6-9557-470c-b1a9-9d0ae8c07581 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:11:00.235984 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d82b6-9557-470c-b1a9-9d0ae8c07581 service nova] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:11:00.236274 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d82b6-9557-470c-b1a9-9d0ae8c07581 service nova] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:11:00.236556 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d82b6-9557-470c-b1a9-9d0ae8c07581 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:11:00.244740 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d82b6-9557-470c-b1a9-9d0ae8c07581 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:11:00.246335 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d82b6-9557-470c-b1a9-9d0ae8c07581 service nova] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:11:00.255757 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 767/1527] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:11:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:11:04.303365 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7f8ed-1e3c-4a38-8c65-3991b26475d8 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:11:04.303935 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7f8ed-1e3c-4a38-8c65-3991b26475d8 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:11:04.303935 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7f8ed-1e3c-4a38-8c65-3991b26475d8 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:11:04.305329 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 761/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 05:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:11:04.311292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf7dc4-5a52-4ce1-bdce-570348e0a297 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:11:04.311701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf7dc4-5a52-4ce1-bdce-570348e0a297 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:11:04.311810 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf7dc4-5a52-4ce1-bdce-570348e0a297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:11:04.331613 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-19cf7dc4-5a52-4ce1-bdce-570348e0a297 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:11:04.332820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-19cf7dc4-5a52-4ce1-bdce-570348e0a297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:11:04.340783 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 768/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 05:11:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:11:04.422098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7f8ed-1e3c-4a38-8c65-3991b26475d8 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:11:04.424288 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-44c7f8ed-1e3c-4a38-8c65-3991b26475d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:11:04.504368 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7f8ed-1e3c-4a38-8c65-3991b26475d8 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:11:04.508050 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163db5bd-10a5-47f5-908d-6dad0fc55f25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:11:04.508658 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163db5bd-10a5-47f5-908d-6dad0fc55f25 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:11:04.509509 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163db5bd-10a5-47f5-908d-6dad0fc55f25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:11:04.509786 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163db5bd-10a5-47f5-908d-6dad0fc55f25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:11:04.519241 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163db5bd-10a5-47f5-908d-6dad0fc55f25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:11:04.520697 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163db5bd-10a5-47f5-908d-6dad0fc55f25 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:11:04.530317 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 762/1530] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:11:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:11:06.541526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74321a13-15ca-4387-a4c9-2cc33b1fa64e None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:11:06.542115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74321a13-15ca-4387-a4c9-2cc33b1fa64e None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:11:06.542115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74321a13-15ca-4387-a4c9-2cc33b1fa64e None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:11:06.543451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 769/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 05:11:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:11:06.548629 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45ae45-ae3a-43a4-9ec6-74f0adaf6af5 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:11:06.549055 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45ae45-ae3a-43a4-9ec6-74f0adaf6af5 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:11:06.549199 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45ae45-ae3a-43a4-9ec6-74f0adaf6af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:11:06.569137 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-1e45ae45-ae3a-43a4-9ec6-74f0adaf6af5 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:11:06.570345 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1e45ae45-ae3a-43a4-9ec6-74f0adaf6af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:11:06.578296 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 763/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 05:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:11:06.614803 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74321a13-15ca-4387-a4c9-2cc33b1fa64e None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:11:06.629469 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74321a13-15ca-4387-a4c9-2cc33b1fa64e None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:11:06.634402 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a6bbe4-ca4c-47cd-af51-8d6f30e26938 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:11:06.635050 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6bbe4-ca4c-47cd-af51-8d6f30e26938 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:11:06.635339 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6bbe4-ca4c-47cd-af51-8d6f30e26938 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:11:06.635642 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6bbe4-ca4c-47cd-af51-8d6f30e26938 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:11:06.645225 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a6bbe4-ca4c-47cd-af51-8d6f30e26938 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:11:06.646683 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a6bbe4-ca4c-47cd-af51-8d6f30e26938 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:11:06.656834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 770/1533] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:11:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:11:10.269673 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6031136-33b6-47ea-8fe3-bb38c5d3b811 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:11:10.270243 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6031136-33b6-47ea-8fe3-bb38c5d3b811 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:11:10.270243 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6031136-33b6-47ea-8fe3-bb38c5d3b811 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:11:10.271661 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 764/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 05:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:11:10.277386 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7b9cd-ca22-419e-a986-da1b42637976 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:11:10.278017 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7b9cd-ca22-419e-a986-da1b42637976 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:11:10.278017 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7b9cd-ca22-419e-a986-da1b42637976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:11:10.297753 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-14a7b9cd-ca22-419e-a986-da1b42637976 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:11:10.298978 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-14a7b9cd-ca22-419e-a986-da1b42637976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:11:10.306774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 771/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 05:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:11:10.608910 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6031136-33b6-47ea-8fe3-bb38c5d3b811 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:11:10.610615 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d6031136-33b6-47ea-8fe3-bb38c5d3b811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:11:10.676273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:11:10.690168 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6031136-33b6-47ea-8fe3-bb38c5d3b811 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:11:10.693704 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92232d25-2a10-488c-b3a4-1ba207aff50e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:11:10.695936 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92232d25-2a10-488c-b3a4-1ba207aff50e service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:11:10.695936 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92232d25-2a10-488c-b3a4-1ba207aff50e service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:11:10.695936 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92232d25-2a10-488c-b3a4-1ba207aff50e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:11:10.704366 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92232d25-2a10-488c-b3a4-1ba207aff50e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:11:10.705853 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92232d25-2a10-488c-b3a4-1ba207aff50e service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:11:10.715401 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 765/1536] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:11:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:11:12.745997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b84e2-b8cc-47e2-928d-0023d2c5385d None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:11:12.746548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b84e2-b8cc-47e2-928d-0023d2c5385d None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:11:12.746548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b84e2-b8cc-47e2-928d-0023d2c5385d None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:11:12.747932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 772/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 05:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:11:12.753023 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa137f8f-99c9-4728-a85c-68be53b5e6f5 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:11:12.753448 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa137f8f-99c9-4728-a85c-68be53b5e6f5 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:11:12.753558 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa137f8f-99c9-4728-a85c-68be53b5e6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:11:12.772609 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-fa137f8f-99c9-4728-a85c-68be53b5e6f5 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:11:12.773806 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-fa137f8f-99c9-4728-a85c-68be53b5e6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:11:12.781963 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 766/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 05:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:11:13.090294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b84e2-b8cc-47e2-928d-0023d2c5385d None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:11:13.092042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-310b84e2-b8cc-47e2-928d-0023d2c5385d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:11:13.105627 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b84e2-b8cc-47e2-928d-0023d2c5385d None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:11:13.109105 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf855314-0120-406d-b9be-fffec1ea120a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:11:13.109712 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf855314-0120-406d-b9be-fffec1ea120a service cinder] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:11:13.109991 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf855314-0120-406d-b9be-fffec1ea120a service cinder] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:11:13.110265 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf855314-0120-406d-b9be-fffec1ea120a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:11:13.120891 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf855314-0120-406d-b9be-fffec1ea120a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:11:13.122350 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf855314-0120-406d-b9be-fffec1ea120a service cinder] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:11:13.133429 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 773/1539] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:11:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:11:22.767815 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=11 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:11:23.127940 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=89 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:12:41.664651 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d12c7b-1269-4c1a-8fc6-258c7ea75a1e None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:41.685351 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d12c7b-1269-4c1a-8fc6-258c7ea75a1e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:41.685351 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d12c7b-1269-4c1a-8fc6-258c7ea75a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:41.692565 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-61d12c7b-1269-4c1a-8fc6-258c7ea75a1e None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:41.694301 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-61d12c7b-1269-4c1a-8fc6-258c7ea75a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:41.705760 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 767/1540] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:41.718065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e809c19-3981-4a38-8b5f-7c13b8c11eb1 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:41.719106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e809c19-3981-4a38-8b5f-7c13b8c11eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:41.719614 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e809c19-3981-4a38-8b5f-7c13b8c11eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:41.749094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-5e809c19-3981-4a38-8b5f-7c13b8c11eb1 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:41.750684 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-5e809c19-3981-4a38-8b5f-7c13b8c11eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:41.762421 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 774/1541] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:41.766709 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d12c7b-1269-4c1a-8fc6-258c7ea75a1e None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:41.781535 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d12c7b-1269-4c1a-8fc6-258c7ea75a1e None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:41.786807 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4973af-963b-4e42-8e63-e4fb4257ceff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:41.788205 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4973af-963b-4e42-8e63-e4fb4257ceff admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:41.788349 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4973af-963b-4e42-8e63-e4fb4257ceff admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:41.788544 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4973af-963b-4e42-8e63-e4fb4257ceff admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:41.789263 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4973af-963b-4e42-8e63-e4fb4257ceff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:12:41.789566 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4973af-963b-4e42-8e63-e4fb4257ceff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:41.791253 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4973af-963b-4e42-8e63-e4fb4257ceff admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:41.805880 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 768/1542] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:12:41.820246 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e809c19-3981-4a38-8b5f-7c13b8c11eb1 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:41.834870 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e809c19-3981-4a38-8b5f-7c13b8c11eb1 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:41.840923 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbcc63-4650-4662-ab4f-27b048c57d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:41.841996 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbcc63-4650-4662-ab4f-27b048c57d82 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:41.842518 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbcc63-4650-4662-ab4f-27b048c57d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:41.843053 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbcc63-4650-4662-ab4f-27b048c57d82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:41.843985 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bbcc63-4650-4662-ab4f-27b048c57d82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:41.846494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bbcc63-4650-4662-ab4f-27b048c57d82 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:41.969488 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 775/1543] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:12:41 2022] POST /identity/v3/projects => generated 393 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:12:41.973682 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4973af-963b-4e42-8e63-e4fb4257ceff admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:41.984843 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4973af-963b-4e42-8e63-e4fb4257ceff admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:42.073923 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19383f6e-fd3f-4d8a-8e70-0009f9e77b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:42.074576 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19383f6e-fd3f-4d8a-8e70-0009f9e77b61 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:42.074876 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19383f6e-fd3f-4d8a-8e70-0009f9e77b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:42.075163 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19383f6e-fd3f-4d8a-8e70-0009f9e77b61 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:42.075984 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19383f6e-fd3f-4d8a-8e70-0009f9e77b61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:42.077881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19383f6e-fd3f-4d8a-8e70-0009f9e77b61 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:42.229947 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 769/1544] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:12:41 2022] POST /identity/v3/users => generated 297 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:12:42.233979 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbcc63-4650-4662-ab4f-27b048c57d82 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:42.248156 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbcc63-4650-4662-ab4f-27b048c57d82 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:42.254404 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7ab99-a4f4-45cf-bec3-06af4ca12041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:42.255134 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7ab99-a4f4-45cf-bec3-06af4ca12041 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:42.255422 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7ab99-a4f4-45cf-bec3-06af4ca12041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:42.255703 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7ab99-a4f4-45cf-bec3-06af4ca12041 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:42.259374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7ab99-a4f4-45cf-bec3-06af4ca12041 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:42.261214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7ab99-a4f4-45cf-bec3-06af4ca12041 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:42.263776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a86f3-db4e-460d-9168-7c2c8712b7a4 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:42.264376 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a86f3-db4e-460d-9168-7c2c8712b7a4 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:42.264898 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a86f3-db4e-460d-9168-7c2c8712b7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:42.273493 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 776/1545] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:12:42.277541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7ab99-a4f4-45cf-bec3-06af4ca12041 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:42.293702 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7ab99-a4f4-45cf-bec3-06af4ca12041 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:42.303869 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-9d1a86f3-db4e-460d-9168-7c2c8712b7a4 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:42.305360 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9d1a86f3-db4e-460d-9168-7c2c8712b7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:42.307005 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae3ad3e-0161-42dd-ade4-d48c65c3a3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:42.308162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3ad3e-0161-42dd-ade4-d48c65c3a3e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:42.308684 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3ad3e-0161-42dd-ade4-d48c65c3a3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:42.309185 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3ad3e-0161-42dd-ade4-d48c65c3a3e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:42.310110 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae3ad3e-0161-42dd-ade4-d48c65c3a3e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:42.311975 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae3ad3e-0161-42dd-ade4-d48c65c3a3e7 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:42.320625 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 770/1546] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:42.333464 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 777/1547] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:12:42.336584 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a86f3-db4e-460d-9168-7c2c8712b7a4 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:42.339812 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d92b23-d8f7-4bb0-b0e5-ae4aea03b65c None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:42.340342 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d92b23-d8f7-4bb0-b0e5-ae4aea03b65c None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:42.340865 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d92b23-d8f7-4bb0-b0e5-ae4aea03b65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:42.351281 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a86f3-db4e-460d-9168-7c2c8712b7a4 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:42.366872 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f0ae6-a6f9-4b58-8329-da4264a2e446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:42.366872 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f0ae6-a6f9-4b58-8329-da4264a2e446 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:42.366872 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f0ae6-a6f9-4b58-8329-da4264a2e446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:42.366872 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f0ae6-a6f9-4b58-8329-da4264a2e446 admin admin] PATH_INFO: `/v3/projects/829c8e80e96d40d682a17185eb4da336/users/bb0fe4cb98e0447cad3e126da1a8de9c/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:42.376543 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-81d92b23-d8f7-4bb0-b0e5-ae4aea03b65c None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:42.378619 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-81d92b23-d8f7-4bb0-b0e5-ae4aea03b65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:42.387437 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f0ae6-a6f9-4b58-8329-da4264a2e446 admin admin] RBAC: Authorizing `identity:create_grant(project_id=829c8e80e96d40d682a17185eb4da336, user_id=bb0fe4cb98e0447cad3e126da1a8de9c, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:42.388711 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: WARNING py.warnings [None req-403f0ae6-a6f9-4b58-8329-da4264a2e446 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 05:12:42.388711 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: warnings.warn(msg) Aug 03 05:12:42.388711 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: Aug 03 05:12:42.390993 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f0ae6-a6f9-4b58-8329-da4264a2e446 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:42.394275 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 778/1548] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:12:42.401585 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d92b23-d8f7-4bb0-b0e5-ae4aea03b65c None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:42.415177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d92b23-d8f7-4bb0-b0e5-ae4aea03b65c None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:42.428375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da036895-c430-4195-a844-bc18a61877a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:42.429698 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da036895-c430-4195-a844-bc18a61877a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:42.430212 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da036895-c430-4195-a844-bc18a61877a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:42.430831 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da036895-c430-4195-a844-bc18a61877a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:42.431806 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da036895-c430-4195-a844-bc18a61877a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:12:42.432560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da036895-c430-4195-a844-bc18a61877a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:42.433823 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: WARNING py.warnings [None req-da036895-c430-4195-a844-bc18a61877a3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 03 05:12:42.433823 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: warnings.warn(msg) Aug 03 05:12:42.433823 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: Aug 03 05:12:42.435161 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da036895-c430-4195-a844-bc18a61877a3 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:42.455734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 779/1549] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 05:12:42.465687 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da036895-c430-4195-a844-bc18a61877a3 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:42.476399 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 771/1550] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:12:42 2022] PUT /identity/v3/projects/829c8e80e96d40d682a17185eb4da336/users/bb0fe4cb98e0447cad3e126da1a8de9c/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:12:42.481023 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da036895-c430-4195-a844-bc18a61877a3 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:42.481160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f24b29-c706-470a-8524-11b07ab788c3 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:42.481527 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f24b29-c706-470a-8524-11b07ab788c3 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:42.482006 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f24b29-c706-470a-8524-11b07ab788c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:42.513979 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-e5f24b29-c706-470a-8524-11b07ab788c3 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:42.515403 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e5f24b29-c706-470a-8524-11b07ab788c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:42.586020 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62094922-33f3-4be9-bfda-a79c808b9b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:42.587061 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62094922-33f3-4be9-bfda-a79c808b9b31 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:42.589696 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62094922-33f3-4be9-bfda-a79c808b9b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:42.590289 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62094922-33f3-4be9-bfda-a79c808b9b31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:42.591201 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62094922-33f3-4be9-bfda-a79c808b9b31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:42.593702 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62094922-33f3-4be9-bfda-a79c808b9b31 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:42.602875 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 772/1551] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:12:42.604256 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e775bb-3b3e-4ac9-9dfe-279890a718ea None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:42.604550 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e775bb-3b3e-4ac9-9dfe-279890a718ea None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:42.604910 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e775bb-3b3e-4ac9-9dfe-279890a718ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:42.635587 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-09e775bb-3b3e-4ac9-9dfe-279890a718ea None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:42.637054 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-09e775bb-3b3e-4ac9-9dfe-279890a718ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:42.648010 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 773/1552] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:42.649248 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3950b-e764-4b41-984e-c80b95ebd017 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:42.649537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3950b-e764-4b41-984e-c80b95ebd017 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:42.649920 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3950b-e764-4b41-984e-c80b95ebd017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:42.671452 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-d4e3950b-e764-4b41-984e-c80b95ebd017 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:42.672691 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d4e3950b-e764-4b41-984e-c80b95ebd017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:42.681485 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 774/1553] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:42.682664 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22884c9-4ea0-44d1-93cb-fe4e88165e46 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:42.682976 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22884c9-4ea0-44d1-93cb-fe4e88165e46 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:42.683255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22884c9-4ea0-44d1-93cb-fe4e88165e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:42.696150 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 780/1554] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:12:42 2022] POST /identity/v3/projects => generated 383 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:12:42.696150 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b3931-c52e-4b34-8a19-997b17e2f2df None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:42.696150 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b3931-c52e-4b34-8a19-997b17e2f2df None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:42.696150 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b3931-c52e-4b34-8a19-997b17e2f2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:42.703049 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-c22884c9-4ea0-44d1-93cb-fe4e88165e46 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:42.704358 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c22884c9-4ea0-44d1-93cb-fe4e88165e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:42.715563 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-926b3931-c52e-4b34-8a19-997b17e2f2df None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:42.716753 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-926b3931-c52e-4b34-8a19-997b17e2f2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:42.726637 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:12:42.765903 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 775/1555] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:42.766708 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22884c9-4ea0-44d1-93cb-fe4e88165e46 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:42.777170 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 781/1556] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:42.778602 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da73ec1-a104-4ec9-b714-cd42d4f33f1d None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:42.778935 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da73ec1-a104-4ec9-b714-cd42d4f33f1d None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:42.779217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da73ec1-a104-4ec9-b714-cd42d4f33f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:42.793809 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:12:42.799692 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-2da73ec1-a104-4ec9-b714-cd42d4f33f1d None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:42.800914 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-2da73ec1-a104-4ec9-b714-cd42d4f33f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:42.809365 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 782/1557] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:42.810719 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e8411-8d02-4c9a-82e4-47f622ebc9b7 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:42.811096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e8411-8d02-4c9a-82e4-47f622ebc9b7 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:42.811380 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e8411-8d02-4c9a-82e4-47f622ebc9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:42.831222 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-262e8411-8d02-4c9a-82e4-47f622ebc9b7 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:42.832437 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-262e8411-8d02-4c9a-82e4-47f622ebc9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:42.837078 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22884c9-4ea0-44d1-93cb-fe4e88165e46 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:42.840751 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6ccba5-1fca-4296-8386-951ff80fbe66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:42.841084 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 783/1558] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:42.841376 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ccba5-1fca-4296-8386-951ff80fbe66 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:42.846568 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ccba5-1fca-4296-8386-951ff80fbe66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:42.846568 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ccba5-1fca-4296-8386-951ff80fbe66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:42.846794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f963ff9-124c-4c91-ac4c-adf7099a5e62 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:42.846794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f963ff9-124c-4c91-ac4c-adf7099a5e62 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:42.846794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f963ff9-124c-4c91-ac4c-adf7099a5e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:42.855734 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6ccba5-1fca-4296-8386-951ff80fbe66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:42.857318 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6ccba5-1fca-4296-8386-951ff80fbe66 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:42.865153 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-9f963ff9-124c-4c91-ac4c-adf7099a5e62 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:42.866417 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-9f963ff9-124c-4c91-ac4c-adf7099a5e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:42.866848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 776/1559] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:12:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:12:42.868127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48840d1-d56e-4361-ad77-cd1e3c02ff6c None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:42.868428 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48840d1-d56e-4361-ad77-cd1e3c02ff6c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:42.868740 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48840d1-d56e-4361-ad77-cd1e3c02ff6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:42.874885 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 784/1560] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:42.875755 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f963ff9-124c-4c91-ac4c-adf7099a5e62 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:42.885333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f963ff9-124c-4c91-ac4c-adf7099a5e62 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:42.890913 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-c48840d1-d56e-4361-ad77-cd1e3c02ff6c None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:42.891504 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd86ed-17b2-4581-a173-1897998608c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:42.892135 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd86ed-17b2-4581-a173-1897998608c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:42.892263 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c48840d1-d56e-4361-ad77-cd1e3c02ff6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:42.892491 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd86ed-17b2-4581-a173-1897998608c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:42.892806 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd86ed-17b2-4581-a173-1897998608c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:42.893820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd86ed-17b2-4581-a173-1897998608c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:42.894879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: WARNING py.warnings [None req-aebd86ed-17b2-4581-a173-1897998608c6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 05:12:42.894879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: warnings.warn(msg) Aug 03 05:12:42.894879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: Aug 03 05:12:42.896142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd86ed-17b2-4581-a173-1897998608c6 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:42.901127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 777/1561] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:42.908477 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07629b24-806a-4693-a178-138283ed8205 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:42.908477 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07629b24-806a-4693-a178-138283ed8205 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:42.908477 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07629b24-806a-4693-a178-138283ed8205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:42.924850 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-07629b24-806a-4693-a178-138283ed8205 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:42.926055 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-07629b24-806a-4693-a178-138283ed8205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:42.934043 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 778/1562] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:42.936982 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb104b-cc28-4810-8f88-3927e5e06aa4 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:42.936982 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb104b-cc28-4810-8f88-3927e5e06aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:42.936982 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb104b-cc28-4810-8f88-3927e5e06aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:42.954733 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-f3cb104b-cc28-4810-8f88-3927e5e06aa4 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:42.955942 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f3cb104b-cc28-4810-8f88-3927e5e06aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:42.968259 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 779/1563] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:42.972934 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12d08d-1ff7-4591-bd38-d52279042c4c None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:42.972934 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12d08d-1ff7-4591-bd38-d52279042c4c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:42.972934 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12d08d-1ff7-4591-bd38-d52279042c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:42.991953 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-1e12d08d-1ff7-4591-bd38-d52279042c4c None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:42.993171 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1e12d08d-1ff7-4591-bd38-d52279042c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:42.995643 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 785/1564] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:12:42 2022] POST /identity/v3/users => generated 292 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:12:42.995643 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52a439-40b6-4593-b55c-3052867409c2 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:43.002400 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52a439-40b6-4593-b55c-3052867409c2 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:43.002400 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52a439-40b6-4593-b55c-3052867409c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:43.008626 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 780/1565] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:43.008626 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12d08d-1ff7-4591-bd38-d52279042c4c None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:43.019443 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12d08d-1ff7-4591-bd38-d52279042c4c None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:43.025067 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3eb77d-7e41-4650-bff1-2f2762ca07a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:43.029345 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3eb77d-7e41-4650-bff1-2f2762ca07a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:43.029474 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-da52a439-40b6-4593-b55c-3052867409c2 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:43.029587 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3eb77d-7e41-4650-bff1-2f2762ca07a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:43.029587 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3eb77d-7e41-4650-bff1-2f2762ca07a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:43.029794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-da52a439-40b6-4593-b55c-3052867409c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:43.029886 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3eb77d-7e41-4650-bff1-2f2762ca07a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:12:43.029886 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3eb77d-7e41-4650-bff1-2f2762ca07a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:43.029886 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3eb77d-7e41-4650-bff1-2f2762ca07a1 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:43.040415 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 786/1566] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:43.040415 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52a439-40b6-4593-b55c-3052867409c2 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:43.047782 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52a439-40b6-4593-b55c-3052867409c2 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:43.051068 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 781/1567] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:12:43.058725 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3eb77d-7e41-4650-bff1-2f2762ca07a1 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:43.058897 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2c2e5-8609-4c30-b53a-113c9c184900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:43.058897 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2c2e5-8609-4c30-b53a-113c9c184900 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:43.058897 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2c2e5-8609-4c30-b53a-113c9c184900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:43.058897 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2c2e5-8609-4c30-b53a-113c9c184900 admin admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:43.059231 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e2c2e5-8609-4c30-b53a-113c9c184900 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:12:43.059231 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e2c2e5-8609-4c30-b53a-113c9c184900 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:43.059231 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e2c2e5-8609-4c30-b53a-113c9c184900 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:43.064735 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3eb77d-7e41-4650-bff1-2f2762ca07a1 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:43.069367 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd868f32-d876-4c08-aef9-a38bb5835d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:43.077892 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 787/1568] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:12:43.078068 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd868f32-d876-4c08-aef9-a38bb5835d40 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:43.078068 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd868f32-d876-4c08-aef9-a38bb5835d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:43.078068 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd868f32-d876-4c08-aef9-a38bb5835d40 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:43.078357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2c2e5-8609-4c30-b53a-113c9c184900 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:43.078483 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd868f32-d876-4c08-aef9-a38bb5835d40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:12:43.078483 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd868f32-d876-4c08-aef9-a38bb5835d40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:43.078483 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd868f32-d876-4c08-aef9-a38bb5835d40 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:43.086039 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 782/1569] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:12:43.087839 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd868f32-d876-4c08-aef9-a38bb5835d40 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:43.088608 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2c2e5-8609-4c30-b53a-113c9c184900 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:43.093708 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989fe53-b7e9-469b-b484-8c6368593819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:43.094933 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989fe53-b7e9-469b-b484-8c6368593819 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:43.095364 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989fe53-b7e9-469b-b484-8c6368593819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:43.095656 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989fe53-b7e9-469b-b484-8c6368593819 admin admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:43.096360 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4989fe53-b7e9-469b-b484-8c6368593819 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:12:43.096667 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4989fe53-b7e9-469b-b484-8c6368593819 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:43.097887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd868f32-d876-4c08-aef9-a38bb5835d40 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:43.098301 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4989fe53-b7e9-469b-b484-8c6368593819 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:43.103773 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5708149c-0f27-4a4d-922e-1a626c29a99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:43.104393 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708149c-0f27-4a4d-922e-1a626c29a99b admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:43.104681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708149c-0f27-4a4d-922e-1a626c29a99b admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:43.104962 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708149c-0f27-4a4d-922e-1a626c29a99b admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:43.105688 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5708149c-0f27-4a4d-922e-1a626c29a99b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:43.107328 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5708149c-0f27-4a4d-922e-1a626c29a99b admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:43.112062 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 788/1570] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:12:43.113021 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989fe53-b7e9-469b-b484-8c6368593819 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:43.117886 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 783/1571] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:12:43.119035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5708149c-0f27-4a4d-922e-1a626c29a99b admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:43.123173 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989fe53-b7e9-469b-b484-8c6368593819 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:43.128188 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2059b-8e6f-4c8c-a43d-3b84abcf1488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:43.128799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2059b-8e6f-4c8c-a43d-3b84abcf1488 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:43.129111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2059b-8e6f-4c8c-a43d-3b84abcf1488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:43.129441 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2059b-8e6f-4c8c-a43d-3b84abcf1488 admin admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:43.129700 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5708149c-0f27-4a4d-922e-1a626c29a99b admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:43.130142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db2059b-8e6f-4c8c-a43d-3b84abcf1488 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:12:43.130549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db2059b-8e6f-4c8c-a43d-3b84abcf1488 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:43.132239 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db2059b-8e6f-4c8c-a43d-3b84abcf1488 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:43.135160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c529d8c-dcfe-4831-a073-a6438af3b338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:43.135877 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c529d8c-dcfe-4831-a073-a6438af3b338 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:43.136189 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c529d8c-dcfe-4831-a073-a6438af3b338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:43.136496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c529d8c-dcfe-4831-a073-a6438af3b338 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:43.137387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c529d8c-dcfe-4831-a073-a6438af3b338 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:12:43.137615 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c529d8c-dcfe-4831-a073-a6438af3b338 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:43.139626 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c529d8c-dcfe-4831-a073-a6438af3b338 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:43.145090 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 789/1572] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:12:43.145766 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2059b-8e6f-4c8c-a43d-3b84abcf1488 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:43.154758 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 784/1573] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:12:43.155555 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c529d8c-dcfe-4831-a073-a6438af3b338 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:43.157444 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2059b-8e6f-4c8c-a43d-3b84abcf1488 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:43.166537 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b7de8c-75b3-42ce-8e2c-6e805762ea4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:43.168928 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c529d8c-dcfe-4831-a073-a6438af3b338 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:43.169057 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7de8c-75b3-42ce-8e2c-6e805762ea4d admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:43.169057 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7de8c-75b3-42ce-8e2c-6e805762ea4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:43.169057 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7de8c-75b3-42ce-8e2c-6e805762ea4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:43.169057 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b7de8c-75b3-42ce-8e2c-6e805762ea4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:43.170290 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b7de8c-75b3-42ce-8e2c-6e805762ea4d admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:43.172810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62376c-cdcc-4f43-bdaa-6c926ab6aa5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:43.173487 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62376c-cdcc-4f43-bdaa-6c926ab6aa5c admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:43.173781 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62376c-cdcc-4f43-bdaa-6c926ab6aa5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:43.174107 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62376c-cdcc-4f43-bdaa-6c926ab6aa5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:43.175085 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62376c-cdcc-4f43-bdaa-6c926ab6aa5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:43.176385 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: WARNING py.warnings [None req-4d62376c-cdcc-4f43-bdaa-6c926ab6aa5c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 05:12:43.176385 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: warnings.warn(msg) Aug 03 05:12:43.176385 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: Aug 03 05:12:43.177706 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62376c-cdcc-4f43-bdaa-6c926ab6aa5c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:43.263105 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 790/1574] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:12:43 2022] POST /identity/v3/projects => generated 375 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:12:43.263105 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b7de8c-75b3-42ce-8e2c-6e805762ea4d admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:43.316332 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 785/1575] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:12:43 2022] POST /identity/v3/projects => generated 367 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:12:43.317282 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62376c-cdcc-4f43-bdaa-6c926ab6aa5c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:43.324385 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b7de8c-75b3-42ce-8e2c-6e805762ea4d admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:43.361346 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:12:43.387406 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0f0b8-a503-4cf3-a524-573a1f1c5292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:43.388535 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0f0b8-a503-4cf3-a524-573a1f1c5292 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:43.389614 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0f0b8-a503-4cf3-a524-573a1f1c5292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:43.390092 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0f0b8-a503-4cf3-a524-573a1f1c5292 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:43.391036 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b0f0b8-a503-4cf3-a524-573a1f1c5292 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:43.393195 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b0f0b8-a503-4cf3-a524-573a1f1c5292 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:43.395157 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62376c-cdcc-4f43-bdaa-6c926ab6aa5c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:43.400774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82d3f1-40fe-4d28-9a30-76eda58b8961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:43.401410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82d3f1-40fe-4d28-9a30-76eda58b8961 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:43.401673 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82d3f1-40fe-4d28-9a30-76eda58b8961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:43.401956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82d3f1-40fe-4d28-9a30-76eda58b8961 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:43.402661 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf82d3f1-40fe-4d28-9a30-76eda58b8961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:43.404407 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf82d3f1-40fe-4d28-9a30-76eda58b8961 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:43.412779 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:12:43.418056 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 786/1576] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:12:43.418784 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82d3f1-40fe-4d28-9a30-76eda58b8961 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:43.428632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82d3f1-40fe-4d28-9a30-76eda58b8961 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:43.433814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd0f31-6812-40d8-9d28-c12bdde5e1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:43.435130 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd0f31-6812-40d8-9d28-c12bdde5e1ac admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:43.435593 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd0f31-6812-40d8-9d28-c12bdde5e1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:43.435878 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd0f31-6812-40d8-9d28-c12bdde5e1ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:43.436541 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdd0f31-6812-40d8-9d28-c12bdde5e1ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:43.438601 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdd0f31-6812-40d8-9d28-c12bdde5e1ac admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:43.502502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 791/1577] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:12:43 2022] POST /identity/v3/projects => generated 371 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:12:43.502502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0f0b8-a503-4cf3-a524-573a1f1c5292 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:43.556814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 787/1578] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:12:43 2022] POST /identity/v3/projects => generated 377 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:12:43.556814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd0f31-6812-40d8-9d28-c12bdde5e1ac admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:43.562644 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0f0b8-a503-4cf3-a524-573a1f1c5292 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:43.628531 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367bb56e-a84c-474f-9e80-4fcf74a3d0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:43.629207 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367bb56e-a84c-474f-9e80-4fcf74a3d0fd admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:43.629464 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367bb56e-a84c-474f-9e80-4fcf74a3d0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:43.629743 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367bb56e-a84c-474f-9e80-4fcf74a3d0fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:43.630276 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd0f31-6812-40d8-9d28-c12bdde5e1ac admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:43.630403 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367bb56e-a84c-474f-9e80-4fcf74a3d0fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:43.633244 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367bb56e-a84c-474f-9e80-4fcf74a3d0fd admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:43.636147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8250b38-e2f3-48f4-99e8-0f3b02a6ef0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:43.636853 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8250b38-e2f3-48f4-99e8-0f3b02a6ef0c admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:43.637211 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8250b38-e2f3-48f4-99e8-0f3b02a6ef0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:43.637560 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8250b38-e2f3-48f4-99e8-0f3b02a6ef0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:43.638747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8250b38-e2f3-48f4-99e8-0f3b02a6ef0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:43.640771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8250b38-e2f3-48f4-99e8-0f3b02a6ef0c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:43.749763 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 792/1579] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:12:43 2022] POST /identity/v3/projects => generated 389 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:12:43.750917 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367bb56e-a84c-474f-9e80-4fcf74a3d0fd admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:43.772597 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 788/1580] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:12:43 2022] POST /identity/v3/projects => generated 379 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:12:43.778328 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8250b38-e2f3-48f4-99e8-0f3b02a6ef0c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:43.816943 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367bb56e-a84c-474f-9e80-4fcf74a3d0fd admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:43.840581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8250b38-e2f3-48f4-99e8-0f3b02a6ef0c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:43.846766 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a6cec-d87a-4ea4-8955-3cfa7436939f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:43.847477 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a6cec-d87a-4ea4-8955-3cfa7436939f admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:43.847761 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a6cec-d87a-4ea4-8955-3cfa7436939f admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:43.848042 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a6cec-d87a-4ea4-8955-3cfa7436939f admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:43.848985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36a6cec-d87a-4ea4-8955-3cfa7436939f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:43.849627 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:12:43.851287 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36a6cec-d87a-4ea4-8955-3cfa7436939f admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:43.879255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ccfb57-12d3-4a16-a3f1-565db817a5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:43.879914 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccfb57-12d3-4a16-a3f1-565db817a5bd admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:43.880200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccfb57-12d3-4a16-a3f1-565db817a5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:43.880497 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccfb57-12d3-4a16-a3f1-565db817a5bd admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:43.881752 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ccfb57-12d3-4a16-a3f1-565db817a5bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:43.883752 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ccfb57-12d3-4a16-a3f1-565db817a5bd admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:43.940970 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 789/1581] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:12:43 2022] POST /identity/v3/users => generated 284 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:12:43.940970 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a6cec-d87a-4ea4-8955-3cfa7436939f admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:43.948295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a6cec-d87a-4ea4-8955-3cfa7436939f admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:43.952932 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1b463-5819-492a-b5b7-b579f7aae4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:43.953572 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1b463-5819-492a-b5b7-b579f7aae4f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:43.953857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1b463-5819-492a-b5b7-b579f7aae4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:43.954137 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1b463-5819-492a-b5b7-b579f7aae4f8 admin admin] PATH_INFO: `/v3/projects/cdd26a4536e04491972ff6618289fc8f/users/3915a35da37a4e5cb8cfbf366f3e50a8/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:43.969275 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a1b463-5819-492a-b5b7-b579f7aae4f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdd26a4536e04491972ff6618289fc8f, user_id=3915a35da37a4e5cb8cfbf366f3e50a8, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:43.972011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a1b463-5819-492a-b5b7-b579f7aae4f8 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:43.975816 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:12:43.998691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 793/1582] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:12:43 2022] POST /identity/v3/users => generated 288 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:12:43.998691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ccfb57-12d3-4a16-a3f1-565db817a5bd admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:44.005752 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ccfb57-12d3-4a16-a3f1-565db817a5bd admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:44.010599 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add9a380-7258-4b05-bf77-84bbf91c6729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:44.011301 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9a380-7258-4b05-bf77-84bbf91c6729 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:44.011647 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9a380-7258-4b05-bf77-84bbf91c6729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:44.011923 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9a380-7258-4b05-bf77-84bbf91c6729 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:44.012625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add9a380-7258-4b05-bf77-84bbf91c6729 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:44.014603 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add9a380-7258-4b05-bf77-84bbf91c6729 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:44.075138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 790/1583] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:12:43 2022] PUT /identity/v3/projects/cdd26a4536e04491972ff6618289fc8f/users/3915a35da37a4e5cb8cfbf366f3e50a8/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:12:44.084248 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1b463-5819-492a-b5b7-b579f7aae4f8 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:44.110508 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 794/1584] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:12:43 2022] POST /identity/v3/users => generated 286 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:12:44.110508 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add9a380-7258-4b05-bf77-84bbf91c6729 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:44.122185 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add9a380-7258-4b05-bf77-84bbf91c6729 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:44.140252 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1b463-5819-492a-b5b7-b579f7aae4f8 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:44.145819 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7093e3-cede-41be-bd53-3ddf2484c3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:44.146635 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7093e3-cede-41be-bd53-3ddf2484c3e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:44.147161 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7093e3-cede-41be-bd53-3ddf2484c3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:44.147450 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7093e3-cede-41be-bd53-3ddf2484c3e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:44.148125 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7093e3-cede-41be-bd53-3ddf2484c3e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:44.150464 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7093e3-cede-41be-bd53-3ddf2484c3e0 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:44.181934 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8cf37-844e-418f-ba4c-bb6be819147c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:44.182579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8cf37-844e-418f-ba4c-bb6be819147c admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:44.182894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8cf37-844e-418f-ba4c-bb6be819147c admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:44.183178 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8cf37-844e-418f-ba4c-bb6be819147c admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:44.183851 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd8cf37-844e-418f-ba4c-bb6be819147c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:44.186344 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd8cf37-844e-418f-ba4c-bb6be819147c admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:44.318451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 795/1585] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:12:44 2022] POST /identity/v3/users => generated 295 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:12:44.319424 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8cf37-844e-418f-ba4c-bb6be819147c admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:44.330111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8cf37-844e-418f-ba4c-bb6be819147c admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:44.333117 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 791/1586] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:12:44 2022] POST /identity/v3/users => generated 289 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:12:44.337047 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7093e3-cede-41be-bd53-3ddf2484c3e0 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:44.339531 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05be30-404d-453b-b6c4-4b38aa5af39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:44.340502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05be30-404d-453b-b6c4-4b38aa5af39b admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:44.340962 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05be30-404d-453b-b6c4-4b38aa5af39b admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:44.341595 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05be30-404d-453b-b6c4-4b38aa5af39b admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:44.342435 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e05be30-404d-453b-b6c4-4b38aa5af39b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:44.344759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e05be30-404d-453b-b6c4-4b38aa5af39b admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:44.354075 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7093e3-cede-41be-bd53-3ddf2484c3e0 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:44.362156 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43c1651-6c7c-49f3-8292-14347d03d9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:44.362886 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c1651-6c7c-49f3-8292-14347d03d9b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:44.363238 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c1651-6c7c-49f3-8292-14347d03d9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:44.363581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c1651-6c7c-49f3-8292-14347d03d9b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:44.364501 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43c1651-6c7c-49f3-8292-14347d03d9b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:44.366345 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43c1651-6c7c-49f3-8292-14347d03d9b3 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:44.377050 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 792/1587] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:12:44.381215 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43c1651-6c7c-49f3-8292-14347d03d9b3 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:44.390228 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43c1651-6c7c-49f3-8292-14347d03d9b3 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:44.395175 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124c8b3-c829-4029-83fd-c9f5559f87c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:44.395789 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124c8b3-c829-4029-83fd-c9f5559f87c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:44.396074 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124c8b3-c829-4029-83fd-c9f5559f87c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:44.396352 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124c8b3-c829-4029-83fd-c9f5559f87c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:44.397573 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124c8b3-c829-4029-83fd-c9f5559f87c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:44.399267 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124c8b3-c829-4029-83fd-c9f5559f87c8 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:44.410459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 793/1588] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:12:44.411802 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b418c22-a72f-42b1-a4b3-b42c26f26a50 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:44.412080 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b418c22-a72f-42b1-a4b3-b42c26f26a50 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:44.412960 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b418c22-a72f-42b1-a4b3-b42c26f26a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:44.440891 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 796/1589] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:12:44 2022] POST /identity/v3/users => generated 290 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:12:44.442004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05be30-404d-453b-b6c4-4b38aa5af39b admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:44.443074 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-3b418c22-a72f-42b1-a4b3-b42c26f26a50 None None] MFA Rules not processed for user `3915a35da37a4e5cb8cfbf366f3e50a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:44.444669 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3b418c22-a72f-42b1-a4b3-b42c26f26a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:44.460437 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05be30-404d-453b-b6c4-4b38aa5af39b admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:44.467416 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca242ec-db86-47b6-816a-39a8ea15205c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:44.468917 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca242ec-db86-47b6-816a-39a8ea15205c admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:44.469299 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca242ec-db86-47b6-816a-39a8ea15205c admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:44.469725 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca242ec-db86-47b6-816a-39a8ea15205c admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:44.470698 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca242ec-db86-47b6-816a-39a8ea15205c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:44.472947 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca242ec-db86-47b6-816a-39a8ea15205c admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:44.492173 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 797/1590] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:12:44.492173 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca242ec-db86-47b6-816a-39a8ea15205c admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:44.503983 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca242ec-db86-47b6-816a-39a8ea15205c admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:44.510177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c80cb9-c977-4c9a-9cca-269cd6fe7022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:44.511226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c80cb9-c977-4c9a-9cca-269cd6fe7022 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:44.511613 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c80cb9-c977-4c9a-9cca-269cd6fe7022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:44.511989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c80cb9-c977-4c9a-9cca-269cd6fe7022 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:44.513391 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c80cb9-c977-4c9a-9cca-269cd6fe7022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:44.515422 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c80cb9-c977-4c9a-9cca-269cd6fe7022 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:44.530494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 798/1591] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:12:44.530696 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c80cb9-c977-4c9a-9cca-269cd6fe7022 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:44.542864 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c80cb9-c977-4c9a-9cca-269cd6fe7022 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:44.543532 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 794/1592] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:44.547107 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b418c22-a72f-42b1-a4b3-b42c26f26a50 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:44.548834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b153a4b5-9f81-4cc9-acad-462cb6847997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:44.549533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153a4b5-9f81-4cc9-acad-462cb6847997 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:44.549899 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153a4b5-9f81-4cc9-acad-462cb6847997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:44.550437 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153a4b5-9f81-4cc9-acad-462cb6847997 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:44.551251 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b153a4b5-9f81-4cc9-acad-462cb6847997 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:44.553646 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b153a4b5-9f81-4cc9-acad-462cb6847997 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:44.556408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b418c22-a72f-42b1-a4b3-b42c26f26a50 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:44.562229 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c40755f-ba63-4795-84ec-455db3910deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:44.564232 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c40755f-ba63-4795-84ec-455db3910deb admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:44.564232 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c40755f-ba63-4795-84ec-455db3910deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:44.564232 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c40755f-ba63-4795-84ec-455db3910deb admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:44.564232 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c40755f-ba63-4795-84ec-455db3910deb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:44.565190 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 799/1593] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:12:44.570488 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b153a4b5-9f81-4cc9-acad-462cb6847997 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:44.570687 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c40755f-ba63-4795-84ec-455db3910deb admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:44.579573 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b153a4b5-9f81-4cc9-acad-462cb6847997 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:44.585745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcceb93-935d-4408-b8a6-52759e4507c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:44.586516 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcceb93-935d-4408-b8a6-52759e4507c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:44.586806 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcceb93-935d-4408-b8a6-52759e4507c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:44.587168 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcceb93-935d-4408-b8a6-52759e4507c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:44.587958 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcceb93-935d-4408-b8a6-52759e4507c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:44.589594 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcceb93-935d-4408-b8a6-52759e4507c0 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:44.591713 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 795/1594] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:12:44.593304 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c40755f-ba63-4795-84ec-455db3910deb admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:44.605240 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 800/1595] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:12:44.605240 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcceb93-935d-4408-b8a6-52759e4507c0 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:44.613043 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcceb93-935d-4408-b8a6-52759e4507c0 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:44.617916 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c40755f-ba63-4795-84ec-455db3910deb admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:44.623067 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5305bb-d10c-4af3-9f66-c2d8b6bde016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:44.623067 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5305bb-d10c-4af3-9f66-c2d8b6bde016 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:44.623067 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5305bb-d10c-4af3-9f66-c2d8b6bde016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:44.623067 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5305bb-d10c-4af3-9f66-c2d8b6bde016 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:44.623499 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5305bb-d10c-4af3-9f66-c2d8b6bde016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:44.623499 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5305bb-d10c-4af3-9f66-c2d8b6bde016 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:44.624765 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a37d1c-82c5-471a-9bba-8e0b48700e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:44.625840 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a37d1c-82c5-471a-9bba-8e0b48700e3e admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:44.626124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a37d1c-82c5-471a-9bba-8e0b48700e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:44.626492 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a37d1c-82c5-471a-9bba-8e0b48700e3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:44.627284 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a37d1c-82c5-471a-9bba-8e0b48700e3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:44.629834 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a37d1c-82c5-471a-9bba-8e0b48700e3e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:44.634631 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 801/1596] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:12:44.635399 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5305bb-d10c-4af3-9f66-c2d8b6bde016 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:44.652068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5305bb-d10c-4af3-9f66-c2d8b6bde016 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:44.654333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 796/1597] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:12:44.656342 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a37d1c-82c5-471a-9bba-8e0b48700e3e admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:44.663166 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ec5fc-2914-4d79-9add-c2aeb8aa8121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:44.663969 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ec5fc-2914-4d79-9add-c2aeb8aa8121 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:44.665368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ec5fc-2914-4d79-9add-c2aeb8aa8121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:44.665716 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ec5fc-2914-4d79-9add-c2aeb8aa8121 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:44.666835 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3ec5fc-2914-4d79-9add-c2aeb8aa8121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:44.668061 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3ec5fc-2914-4d79-9add-c2aeb8aa8121 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:44.672818 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a37d1c-82c5-471a-9bba-8e0b48700e3e admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:44.678416 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d462c4-8ed5-4bea-87c9-916fd55c76f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:44.679151 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d462c4-8ed5-4bea-87c9-916fd55c76f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:44.679410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d462c4-8ed5-4bea-87c9-916fd55c76f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:44.679538 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 802/1598] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:12:44.679648 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d462c4-8ed5-4bea-87c9-916fd55c76f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:44.680217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ec5fc-2914-4d79-9add-c2aeb8aa8121 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:44.680339 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d462c4-8ed5-4bea-87c9-916fd55c76f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:44.682182 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d462c4-8ed5-4bea-87c9-916fd55c76f2 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:44.690741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ec5fc-2914-4d79-9add-c2aeb8aa8121 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:44.697475 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2709e2ec-2f59-4c60-bbd3-911038623edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:44.698111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709e2ec-2f59-4c60-bbd3-911038623edc admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:44.698393 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709e2ec-2f59-4c60-bbd3-911038623edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:44.698967 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709e2ec-2f59-4c60-bbd3-911038623edc admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:44.700217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2709e2ec-2f59-4c60-bbd3-911038623edc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:44.702014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2709e2ec-2f59-4c60-bbd3-911038623edc admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:44.708408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:12:44.718050 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 803/1599] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:12:44.718050 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2709e2ec-2f59-4c60-bbd3-911038623edc admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:44.727066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2709e2ec-2f59-4c60-bbd3-911038623edc admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:44.731738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af791514-2e6a-4a57-aa25-630ca6f70491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:44.732405 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af791514-2e6a-4a57-aa25-630ca6f70491 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:44.732693 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af791514-2e6a-4a57-aa25-630ca6f70491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:44.732973 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af791514-2e6a-4a57-aa25-630ca6f70491 admin admin] PATH_INFO: `/v3/projects/9cd419ba699d4449a25fd90a89de214d/users/e7e15d7a479d40f0aa45226e765ecbaf/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:44.749386 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af791514-2e6a-4a57-aa25-630ca6f70491 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cd419ba699d4449a25fd90a89de214d, user_id=e7e15d7a479d40f0aa45226e765ecbaf, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:44.752451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af791514-2e6a-4a57-aa25-630ca6f70491 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:44.793729 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 797/1600] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:12:44 2022] POST /identity/v3/projects => generated 379 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:12:44.793729 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d462c4-8ed5-4bea-87c9-916fd55c76f2 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:44.801993 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:12:44.841551 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 804/1601] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:12:44 2022] PUT /identity/v3/projects/9cd419ba699d4449a25fd90a89de214d/users/e7e15d7a479d40f0aa45226e765ecbaf/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:12:44.842512 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af791514-2e6a-4a57-aa25-630ca6f70491 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:44.874099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d462c4-8ed5-4bea-87c9-916fd55c76f2 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:44.916395 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af791514-2e6a-4a57-aa25-630ca6f70491 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:44.921883 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b3c0a-cd9e-4362-89ac-19ae56a2859b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:44.922567 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b3c0a-cd9e-4362-89ac-19ae56a2859b admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:44.922918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b3c0a-cd9e-4362-89ac-19ae56a2859b admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:44.923231 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b3c0a-cd9e-4362-89ac-19ae56a2859b admin admin] PATH_INFO: `/v3/projects/078e30297f6446e0b4860ae523c2cf1f/users/536e72a4b4c540df860ba3e2b3ee6081/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:44.941831 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b3c0a-cd9e-4362-89ac-19ae56a2859b admin admin] RBAC: Authorizing `identity:create_grant(project_id=078e30297f6446e0b4860ae523c2cf1f, user_id=536e72a4b4c540df860ba3e2b3ee6081, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:44.945180 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef9216-ec83-4e8a-9bc7-26ffb779caf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:44.945778 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b3c0a-cd9e-4362-89ac-19ae56a2859b admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:44.945884 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef9216-ec83-4e8a-9bc7-26ffb779caf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:44.946136 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef9216-ec83-4e8a-9bc7-26ffb779caf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:44.946568 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef9216-ec83-4e8a-9bc7-26ffb779caf6 admin admin] PATH_INFO: `/v3/projects/2e1ed25abd8e43009dd8f3f4b7ec1864/users/f01737ba3200470c8635f335d0338ac8/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:44.964355 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef9216-ec83-4e8a-9bc7-26ffb779caf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e1ed25abd8e43009dd8f3f4b7ec1864, user_id=f01737ba3200470c8635f335d0338ac8, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:44.967105 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef9216-ec83-4e8a-9bc7-26ffb779caf6 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:45.073301 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 798/1602] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:12:44 2022] PUT /identity/v3/projects/2e1ed25abd8e43009dd8f3f4b7ec1864/users/f01737ba3200470c8635f335d0338ac8/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:12:45.076102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef9216-ec83-4e8a-9bc7-26ffb779caf6 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:45.083940 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 805/1603] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:12:44 2022] PUT /identity/v3/projects/078e30297f6446e0b4860ae523c2cf1f/users/536e72a4b4c540df860ba3e2b3ee6081/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:12:45.083940 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b3c0a-cd9e-4362-89ac-19ae56a2859b admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:45.169406 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef9216-ec83-4e8a-9bc7-26ffb779caf6 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:45.177311 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8234436-94a9-4d97-a1b6-d0c2f788e890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:45.180278 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8234436-94a9-4d97-a1b6-d0c2f788e890 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:45.180278 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8234436-94a9-4d97-a1b6-d0c2f788e890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:45.180278 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8234436-94a9-4d97-a1b6-d0c2f788e890 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:45.180278 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8234436-94a9-4d97-a1b6-d0c2f788e890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:45.180722 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b3c0a-cd9e-4362-89ac-19ae56a2859b admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:45.180877 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8234436-94a9-4d97-a1b6-d0c2f788e890 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:45.187931 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980140a5-f977-4f14-87b9-cbc77c7352e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:45.188748 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980140a5-f977-4f14-87b9-cbc77c7352e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:45.189095 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980140a5-f977-4f14-87b9-cbc77c7352e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:45.189385 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980140a5-f977-4f14-87b9-cbc77c7352e4 admin admin] PATH_INFO: `/v3/projects/050592077e0b4d26a1555d2daa7f460d/users/747a608061504d7693f0f99ecdfa476e/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:45.196036 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 799/1604] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:12:45.196036 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8234436-94a9-4d97-a1b6-d0c2f788e890 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:45.211136 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8234436-94a9-4d97-a1b6-d0c2f788e890 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:45.215715 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980140a5-f977-4f14-87b9-cbc77c7352e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=050592077e0b4d26a1555d2daa7f460d, user_id=747a608061504d7693f0f99ecdfa476e, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:45.218443 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980140a5-f977-4f14-87b9-cbc77c7352e4 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:45.223149 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc9a323-5df4-4eea-9a98-3019753f58f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:45.223842 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9a323-5df4-4eea-9a98-3019753f58f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:45.224122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9a323-5df4-4eea-9a98-3019753f58f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:45.226140 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9a323-5df4-4eea-9a98-3019753f58f3 admin admin] PATH_INFO: `/v3/projects/ec7ef948182a457381fd0a686ca298dc/users/41541854b74c49a886db2e7bfa867648/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:45.246198 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc9a323-5df4-4eea-9a98-3019753f58f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec7ef948182a457381fd0a686ca298dc, user_id=41541854b74c49a886db2e7bfa867648, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:45.249226 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc9a323-5df4-4eea-9a98-3019753f58f3 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:45.367246 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 800/1606] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:12:45 2022] PUT /identity/v3/projects/ec7ef948182a457381fd0a686ca298dc/users/41541854b74c49a886db2e7bfa867648/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:12:45.368435 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc9a323-5df4-4eea-9a98-3019753f58f3 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:45.370904 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 806/1606] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:12:45 2022] PUT /identity/v3/projects/050592077e0b4d26a1555d2daa7f460d/users/747a608061504d7693f0f99ecdfa476e/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:12:45.376708 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30842467-d492-4184-86db-83ca5eb2c186 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:45.376708 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30842467-d492-4184-86db-83ca5eb2c186 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:45.376708 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30842467-d492-4184-86db-83ca5eb2c186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:45.386673 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:12:45.416418 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-30842467-d492-4184-86db-83ca5eb2c186 None None] MFA Rules not processed for user `e7e15d7a479d40f0aa45226e765ecbaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:45.417670 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-30842467-d492-4184-86db-83ca5eb2c186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:45.452674 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc9a323-5df4-4eea-9a98-3019753f58f3 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:45.460305 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:12:45.496040 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 807/1607] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:45.497496 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fbb93-1eb3-4c00-815d-059bc7e59f6e None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:45.497804 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fbb93-1eb3-4c00-815d-059bc7e59f6e None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:45.498081 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fbb93-1eb3-4c00-815d-059bc7e59f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:45.513102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72485bf-d357-428c-a9d0-73687da54978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:45.513641 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72485bf-d357-428c-a9d0-73687da54978 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:45.513921 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72485bf-d357-428c-a9d0-73687da54978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:45.514197 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72485bf-d357-428c-a9d0-73687da54978 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:45.514892 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72485bf-d357-428c-a9d0-73687da54978 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:45.516806 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72485bf-d357-428c-a9d0-73687da54978 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:45.518742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-d39fbb93-1eb3-4c00-815d-059bc7e59f6e None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:45.519985 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-d39fbb93-1eb3-4c00-815d-059bc7e59f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:45.529465 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 808/1608] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:45.530793 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39720bbb-49ca-4f37-96db-f4bf8a1c3b06 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:45.531170 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39720bbb-49ca-4f37-96db-f4bf8a1c3b06 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:45.533889 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39720bbb-49ca-4f37-96db-f4bf8a1c3b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:45.566404 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-39720bbb-49ca-4f37-96db-f4bf8a1c3b06 None None] MFA Rules not processed for user `536e72a4b4c540df860ba3e2b3ee6081`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:45.567673 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-39720bbb-49ca-4f37-96db-f4bf8a1c3b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:45.634653 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 801/1609] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:12:45 2022] POST /identity/v3/users => generated 296 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:12:45.635747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72485bf-d357-428c-a9d0-73687da54978 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:45.642351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 809/1610] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:45.642351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d43ab1-ed28-4fac-aa6e-461396290175 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:45.642351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d43ab1-ed28-4fac-aa6e-461396290175 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:45.642351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d43ab1-ed28-4fac-aa6e-461396290175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:45.650010 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72485bf-d357-428c-a9d0-73687da54978 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:45.656438 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ad37c-7966-458d-a0b1-666276a8163b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:45.657384 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ad37c-7966-458d-a0b1-666276a8163b admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:45.657584 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ad37c-7966-458d-a0b1-666276a8163b admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:45.657941 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ad37c-7966-458d-a0b1-666276a8163b admin admin] PATH_INFO: `/v3/projects/42d98a9a72b142729e08ab431290358f/users/77af257997f44fb5b5223c5d9af87b00/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:45.662366 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-65d43ab1-ed28-4fac-aa6e-461396290175 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:45.665575 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-65d43ab1-ed28-4fac-aa6e-461396290175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:45.672179 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 810/1611] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:45.677399 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14ce6b-ad48-4b55-9dd6-7882cce855b1 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:45.678077 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14ce6b-ad48-4b55-9dd6-7882cce855b1 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:45.678634 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14ce6b-ad48-4b55-9dd6-7882cce855b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:45.682553 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3ad37c-7966-458d-a0b1-666276a8163b admin admin] RBAC: Authorizing `identity:create_grant(project_id=42d98a9a72b142729e08ab431290358f, user_id=77af257997f44fb5b5223c5d9af87b00, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:45.686360 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3ad37c-7966-458d-a0b1-666276a8163b admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:45.700881 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-0c14ce6b-ad48-4b55-9dd6-7882cce855b1 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:45.702128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-0c14ce6b-ad48-4b55-9dd6-7882cce855b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:45.710541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 811/1612] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:45.711376 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14ce6b-ad48-4b55-9dd6-7882cce855b1 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:45.771629 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 802/1613] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:12:45 2022] PUT /identity/v3/projects/42d98a9a72b142729e08ab431290358f/users/77af257997f44fb5b5223c5d9af87b00/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:12:45.773190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ad37c-7966-458d-a0b1-666276a8163b admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:45.779221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14ce6b-ad48-4b55-9dd6-7882cce855b1 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:45.784027 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8d3ad37c-7966-458d-a0b1-666276a8163b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:45.843966 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ea422-534d-4477-8b93-6e17189daaa7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:45.845133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ea422-534d-4477-8b93-6e17189daaa7 service cinder] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:45.845434 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ea422-534d-4477-8b93-6e17189daaa7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:45.845787 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ea422-534d-4477-8b93-6e17189daaa7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:45.858256 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:12:45.862902 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ad37c-7966-458d-a0b1-666276a8163b admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:45.867806 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510acc6a-8c77-4d70-a962-ba8681ce504a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:45.872341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510acc6a-8c77-4d70-a962-ba8681ce504a service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:45.872630 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510acc6a-8c77-4d70-a962-ba8681ce504a service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:45.872910 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510acc6a-8c77-4d70-a962-ba8681ce504a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:45.930537 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ea422-534d-4477-8b93-6e17189daaa7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:45.932903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ea422-534d-4477-8b93-6e17189daaa7 service cinder] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:45.938319 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:12:45.945004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 812/1614] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:12:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:12:45.950088 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ea422-534d-4477-8b93-6e17189daaa7 service cinder] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:45.973713 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510acc6a-8c77-4d70-a962-ba8681ce504a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:45.976085 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510acc6a-8c77-4d70-a962-ba8681ce504a service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:45.998702 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 803/1615] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:12:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:12:45.999586 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510acc6a-8c77-4d70-a962-ba8681ce504a service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:46.014891 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510acc6a-8c77-4d70-a962-ba8681ce504a service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:46.022771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f5898-c7c3-4525-a00b-4fbb1ff9e7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:46.024234 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f5898-c7c3-4525-a00b-4fbb1ff9e7a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:46.025040 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f5898-c7c3-4525-a00b-4fbb1ff9e7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:46.025476 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f5898-c7c3-4525-a00b-4fbb1ff9e7a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:46.026323 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28f5898-c7c3-4525-a00b-4fbb1ff9e7a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:46.028753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28f5898-c7c3-4525-a00b-4fbb1ff9e7a6 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:46.057419 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ea422-534d-4477-8b93-6e17189daaa7 service cinder] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:46.073445 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99847e43-9fb3-4fcb-a141-586daba17a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:46.074397 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99847e43-9fb3-4fcb-a141-586daba17a4b admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:46.074669 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99847e43-9fb3-4fcb-a141-586daba17a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:46.074980 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99847e43-9fb3-4fcb-a141-586daba17a4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:46.075692 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99847e43-9fb3-4fcb-a141-586daba17a4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:46.079080 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99847e43-9fb3-4fcb-a141-586daba17a4b admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:46.091711 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 813/1616] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:12:46.092621 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99847e43-9fb3-4fcb-a141-586daba17a4b admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:46.104917 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99847e43-9fb3-4fcb-a141-586daba17a4b admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:46.109104 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687638b-2767-464f-aa37-1a5a9cbffeb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:46.109729 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687638b-2767-464f-aa37-1a5a9cbffeb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:46.110016 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687638b-2767-464f-aa37-1a5a9cbffeb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:46.110296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687638b-2767-464f-aa37-1a5a9cbffeb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:46.124109 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c687638b-2767-464f-aa37-1a5a9cbffeb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:46.130113 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c687638b-2767-464f-aa37-1a5a9cbffeb9 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:46.141603 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 814/1617] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:12:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:12:46.142501 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687638b-2767-464f-aa37-1a5a9cbffeb9 service neutron] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:46.153561 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687638b-2767-464f-aa37-1a5a9cbffeb9 service neutron] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:46.158117 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4e5ac-a8a0-434d-83f0-bca232618f7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:46.159697 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4e5ac-a8a0-434d-83f0-bca232618f7e service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:46.159697 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4e5ac-a8a0-434d-83f0-bca232618f7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:46.159697 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4e5ac-a8a0-434d-83f0-bca232618f7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:46.183000 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b4e5ac-a8a0-434d-83f0-bca232618f7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:46.189409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 804/1618] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:12:45 2022] POST /identity/v3/projects => generated 371 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:12:46.191056 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e57447-550d-40e3-b788-4e28e98cb59b None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:46.192362 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e57447-550d-40e3-b788-4e28e98cb59b None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:46.192646 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e57447-550d-40e3-b788-4e28e98cb59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:46.195107 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b4e5ac-a8a0-434d-83f0-bca232618f7e service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:46.223501 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 815/1619] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:12:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:12:46.225129 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4e5ac-a8a0-434d-83f0-bca232618f7e service neutron] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:46.243302 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4e5ac-a8a0-434d-83f0-bca232618f7e service neutron] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:46.275956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-30e57447-550d-40e3-b788-4e28e98cb59b None None] MFA Rules not processed for user `77af257997f44fb5b5223c5d9af87b00`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:46.275956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-30e57447-550d-40e3-b788-4e28e98cb59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:46.320275 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee0c95-da94-4b3a-ac1a-74b88a670816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:46.321046 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee0c95-da94-4b3a-ac1a-74b88a670816 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:46.321343 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee0c95-da94-4b3a-ac1a-74b88a670816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:46.321804 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee0c95-da94-4b3a-ac1a-74b88a670816 admin admin] PATH_INFO: `/v3/projects/ebbe2fc1c4b34d4e9324555109c39c57/users/352bcbfb3ffa4ba8b5109abb9adf45c4/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:46.345921 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee0c95-da94-4b3a-ac1a-74b88a670816 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebbe2fc1c4b34d4e9324555109c39c57, user_id=352bcbfb3ffa4ba8b5109abb9adf45c4, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:46.351578 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee0c95-da94-4b3a-ac1a-74b88a670816 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:46.420758 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 805/1620] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:46.422930 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e57447-550d-40e3-b788-4e28e98cb59b None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:46.441202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e57447-550d-40e3-b788-4e28e98cb59b None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:46.441202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d79a59-a5ab-41fd-b3a5-bfbfb8299ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:46.441202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d79a59-a5ab-41fd-b3a5-bfbfb8299ab9 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:46.441654 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d79a59-a5ab-41fd-b3a5-bfbfb8299ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:46.442013 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d79a59-a5ab-41fd-b3a5-bfbfb8299ab9 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:46.442910 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d79a59-a5ab-41fd-b3a5-bfbfb8299ab9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:46.445581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d79a59-a5ab-41fd-b3a5-bfbfb8299ab9 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:46.469779 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 816/1621] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:12:46 2022] PUT /identity/v3/projects/ebbe2fc1c4b34d4e9324555109c39c57/users/352bcbfb3ffa4ba8b5109abb9adf45c4/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:12:46.471721 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee0c95-da94-4b3a-ac1a-74b88a670816 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:46.485799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:12:46.589171 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 806/1622] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:12:46 2022] POST /identity/v3/users => generated 292 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:12:46.599679 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d79a59-a5ab-41fd-b3a5-bfbfb8299ab9 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:46.644419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d79a59-a5ab-41fd-b3a5-bfbfb8299ab9 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:46.683975 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee0c95-da94-4b3a-ac1a-74b88a670816 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:46.690833 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd4e9b-39ea-439d-875a-c86363eb05e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:46.691612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd4e9b-39ea-439d-875a-c86363eb05e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:46.691901 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd4e9b-39ea-439d-875a-c86363eb05e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:46.692182 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd4e9b-39ea-439d-875a-c86363eb05e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:46.692928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbd4e9b-39ea-439d-875a-c86363eb05e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:46.697395 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbd4e9b-39ea-439d-875a-c86363eb05e2 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:46.795501 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b64e3-f2e3-4b29-9e69-56b394df091f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:46.797878 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b64e3-f2e3-4b29-9e69-56b394df091f admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:46.797878 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b64e3-f2e3-4b29-9e69-56b394df091f admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:46.797878 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b64e3-f2e3-4b29-9e69-56b394df091f admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:46.797878 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b64e3-f2e3-4b29-9e69-56b394df091f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:46.800597 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b64e3-f2e3-4b29-9e69-56b394df091f admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:46.813243 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 807/1623] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:12:46.814068 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b64e3-f2e3-4b29-9e69-56b394df091f admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:46.828184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b64e3-f2e3-4b29-9e69-56b394df091f admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:46.833444 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 817/1624] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:12:46 2022] POST /identity/v3/projects => generated 379 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:12:46.833570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd4e9b-39ea-439d-875a-c86363eb05e2 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:46.834188 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6a3f6-455e-4d75-8d07-36c28c298e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:46.835019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6a3f6-455e-4d75-8d07-36c28c298e88 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:46.835399 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6a3f6-455e-4d75-8d07-36c28c298e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:46.835821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6a3f6-455e-4d75-8d07-36c28c298e88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:46.836613 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa6a3f6-455e-4d75-8d07-36c28c298e88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:46.841592 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa6a3f6-455e-4d75-8d07-36c28c298e88 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:46.869639 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 808/1625] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:12:46.872726 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6a3f6-455e-4d75-8d07-36c28c298e88 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:46.894806 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6a3f6-455e-4d75-8d07-36c28c298e88 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:46.902153 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:12:46.946312 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd4e9b-39ea-439d-875a-c86363eb05e2 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:46.951938 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8561b-951b-41d6-a543-38bb69512951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:46.952699 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8561b-951b-41d6-a543-38bb69512951 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:46.953029 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8561b-951b-41d6-a543-38bb69512951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:46.953604 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8561b-951b-41d6-a543-38bb69512951 admin admin] PATH_INFO: `/v3/projects/ebbe2fc1c4b34d4e9324555109c39c57/users/352bcbfb3ffa4ba8b5109abb9adf45c4/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:46.957012 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f8561b-951b-41d6-a543-38bb69512951 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebbe2fc1c4b34d4e9324555109c39c57, user_id=352bcbfb3ffa4ba8b5109abb9adf45c4, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:46.960447 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f8561b-951b-41d6-a543-38bb69512951 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:46.977136 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 818/1626] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:12:46 2022] PUT /identity/v3/projects/ebbe2fc1c4b34d4e9324555109c39c57/users/352bcbfb3ffa4ba8b5109abb9adf45c4/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:12:46.987300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8561b-951b-41d6-a543-38bb69512951 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:47.031411 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45908a6c-fc26-4889-8840-710d93335889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:47.034361 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45908a6c-fc26-4889-8840-710d93335889 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:47.034361 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45908a6c-fc26-4889-8840-710d93335889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:47.034361 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45908a6c-fc26-4889-8840-710d93335889 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:47.034361 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45908a6c-fc26-4889-8840-710d93335889 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:47.047653 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45908a6c-fc26-4889-8840-710d93335889 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:47.160084 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8561b-951b-41d6-a543-38bb69512951 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:47.164142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 809/1627] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:12:46 2022] POST /identity/v3/users => generated 296 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:12:47.167218 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080786e-5bb4-4e68-8651-bddb68813f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:47.168299 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080786e-5bb4-4e68-8651-bddb68813f9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:47.169128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080786e-5bb4-4e68-8651-bddb68813f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:47.169681 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080786e-5bb4-4e68-8651-bddb68813f9a admin admin] PATH_INFO: `/v3/projects/346d41c01e174b19875cd4c6f14cca0c/users/0b262d10633b415b8ef593e434be58f0/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:47.171871 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e30e5f-3d4f-49c3-a5bc-b3ea59e5c1b4 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:47.175592 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e30e5f-3d4f-49c3-a5bc-b3ea59e5c1b4 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:47.175592 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e30e5f-3d4f-49c3-a5bc-b3ea59e5c1b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:47.189413 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d080786e-5bb4-4e68-8651-bddb68813f9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=346d41c01e174b19875cd4c6f14cca0c, user_id=0b262d10633b415b8ef593e434be58f0, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:47.198929 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d080786e-5bb4-4e68-8651-bddb68813f9a admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:47.218570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-03e30e5f-3d4f-49c3-a5bc-b3ea59e5c1b4 None None] MFA Rules not processed for user `352bcbfb3ffa4ba8b5109abb9adf45c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:47.219984 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-03e30e5f-3d4f-49c3-a5bc-b3ea59e5c1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:47.363284 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 819/1628] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:12:46 2022] PUT /identity/v3/projects/346d41c01e174b19875cd4c6f14cca0c/users/0b262d10633b415b8ef593e434be58f0/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:12:47.364320 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080786e-5bb4-4e68-8651-bddb68813f9a admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:47.390933 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 810/1629] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:47 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:47.397726 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e30e5f-3d4f-49c3-a5bc-b3ea59e5c1b4 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:47.417431 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:12:47.435676 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e30e5f-3d4f-49c3-a5bc-b3ea59e5c1b4 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:47.562743 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0f652-2cb9-443b-b896-3ab70c3e25c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:47.563785 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0f652-2cb9-443b-b896-3ab70c3e25c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:47.564222 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0f652-2cb9-443b-b896-3ab70c3e25c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:47.564614 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0f652-2cb9-443b-b896-3ab70c3e25c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:47.565341 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080786e-5bb4-4e68-8651-bddb68813f9a admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:47.565694 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0f652-2cb9-443b-b896-3ab70c3e25c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:47.568629 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0f652-2cb9-443b-b896-3ab70c3e25c5 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:47.577706 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0d31d3-f0d8-4dd5-8a17-967f5880a9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:47.578936 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d31d3-f0d8-4dd5-8a17-967f5880a9a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:47.579327 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d31d3-f0d8-4dd5-8a17-967f5880a9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:47.579607 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d31d3-f0d8-4dd5-8a17-967f5880a9a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:47.581015 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0d31d3-f0d8-4dd5-8a17-967f5880a9a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:47.586964 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0d31d3-f0d8-4dd5-8a17-967f5880a9a8 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:47.587649 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 811/1630] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:12:47.588762 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0f652-2cb9-443b-b896-3ab70c3e25c5 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:47.622927 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 820/1631] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:12:47.624570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0d31d3-f0d8-4dd5-8a17-967f5880a9a8 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:47.658459 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0d31d3-f0d8-4dd5-8a17-967f5880a9a8 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:47.680982 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf252ec-c8ac-409a-8291-c56e81e716d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:47.682072 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf252ec-c8ac-409a-8291-c56e81e716d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:47.682732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf252ec-c8ac-409a-8291-c56e81e716d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:47.683314 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf252ec-c8ac-409a-8291-c56e81e716d8 admin admin] PATH_INFO: `/v3/projects/346d41c01e174b19875cd4c6f14cca0c/users/0b262d10633b415b8ef593e434be58f0/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:47.686196 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf252ec-c8ac-409a-8291-c56e81e716d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=346d41c01e174b19875cd4c6f14cca0c, user_id=0b262d10633b415b8ef593e434be58f0, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:47.689423 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf252ec-c8ac-409a-8291-c56e81e716d8 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:47.706819 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 821/1632] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:12:47 2022] PUT /identity/v3/projects/346d41c01e174b19875cd4c6f14cca0c/users/0b262d10633b415b8ef593e434be58f0/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:12:47.708115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf252ec-c8ac-409a-8291-c56e81e716d8 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:47.746034 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0f652-2cb9-443b-b896-3ab70c3e25c5 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:47.799867 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:12:47.857495 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f4778-15d9-4376-84d8-96b826acb3f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:47.858135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f4778-15d9-4376-84d8-96b826acb3f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:47.858580 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f4778-15d9-4376-84d8-96b826acb3f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:47.859060 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f4778-15d9-4376-84d8-96b826acb3f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:47.866956 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf252ec-c8ac-409a-8291-c56e81e716d8 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:47.875428 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddbd1ff-c7ed-4748-896d-b84ad0456a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:47.876585 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddbd1ff-c7ed-4748-896d-b84ad0456a06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:47.877689 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddbd1ff-c7ed-4748-896d-b84ad0456a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:47.878644 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddbd1ff-c7ed-4748-896d-b84ad0456a06 admin admin] PATH_INFO: `/v3/projects/5eaf5c3022d449748376965ad6ab87f2/users/cf1b9c109e924ce8899647ebe7a589da/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:47.914182 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddbd1ff-c7ed-4748-896d-b84ad0456a06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eaf5c3022d449748376965ad6ab87f2, user_id=cf1b9c109e924ce8899647ebe7a589da, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:47.916976 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddbd1ff-c7ed-4748-896d-b84ad0456a06 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:47.946537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f4778-15d9-4376-84d8-96b826acb3f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:47.950465 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f4778-15d9-4376-84d8-96b826acb3f0 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:47.967925 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 812/1633] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:12:47 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:12:47.970726 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca1dfb9-f780-43f6-9d0b-bd26a956ff33 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:47.971319 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca1dfb9-f780-43f6-9d0b-bd26a956ff33 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:47.971811 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca1dfb9-f780-43f6-9d0b-bd26a956ff33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:48.030825 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 822/1634] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:12:47 2022] PUT /identity/v3/projects/5eaf5c3022d449748376965ad6ab87f2/users/cf1b9c109e924ce8899647ebe7a589da/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:12:48.032531 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddbd1ff-c7ed-4748-896d-b84ad0456a06 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:48.041259 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-eca1dfb9-f780-43f6-9d0b-bd26a956ff33 None None] MFA Rules not processed for user `0b262d10633b415b8ef593e434be58f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:48.047559 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-eca1dfb9-f780-43f6-9d0b-bd26a956ff33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:48.125969 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddbd1ff-c7ed-4748-896d-b84ad0456a06 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:48.131373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4060cf-c6c6-44d1-a885-47549e985cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:48.132106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4060cf-c6c6-44d1-a885-47549e985cba admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:48.132454 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4060cf-c6c6-44d1-a885-47549e985cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:48.132796 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4060cf-c6c6-44d1-a885-47549e985cba admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:48.134430 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4060cf-c6c6-44d1-a885-47549e985cba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:48.136533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4060cf-c6c6-44d1-a885-47549e985cba admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:48.149481 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 823/1635] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:12:48.153679 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4060cf-c6c6-44d1-a885-47549e985cba admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:48.169354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4060cf-c6c6-44d1-a885-47549e985cba admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:48.181353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8acb6-b093-4cdf-bf62-f17119b769d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:48.181353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8acb6-b093-4cdf-bf62-f17119b769d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:48.181353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8acb6-b093-4cdf-bf62-f17119b769d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:48.181353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8acb6-b093-4cdf-bf62-f17119b769d5 admin admin] PATH_INFO: `/v3/projects/5eaf5c3022d449748376965ad6ab87f2/users/cf1b9c109e924ce8899647ebe7a589da/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:48.182730 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b8acb6-b093-4cdf-bf62-f17119b769d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eaf5c3022d449748376965ad6ab87f2, user_id=cf1b9c109e924ce8899647ebe7a589da, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:48.187526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b8acb6-b093-4cdf-bf62-f17119b769d5 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:48.207126 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 824/1636] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:12:48 2022] PUT /identity/v3/projects/5eaf5c3022d449748376965ad6ab87f2/users/cf1b9c109e924ce8899647ebe7a589da/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:12:48.208305 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8acb6-b093-4cdf-bf62-f17119b769d5 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:48.236463 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 813/1637] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:47 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:48.236463 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73d9e6-7e21-4dc9-a272-5eeace960414 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:48.236463 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73d9e6-7e21-4dc9-a272-5eeace960414 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:48.236463 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73d9e6-7e21-4dc9-a272-5eeace960414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:48.604176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-2b73d9e6-7e21-4dc9-a272-5eeace960414 None None] MFA Rules not processed for user `cf1b9c109e924ce8899647ebe7a589da`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:48.604176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2b73d9e6-7e21-4dc9-a272-5eeace960414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:48.606597 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8acb6-b093-4cdf-bf62-f17119b769d5 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:48.606597 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135fec94-457c-4c09-a796-55cb409b775c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:48.606597 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fec94-457c-4c09-a796-55cb409b775c service cinder] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:48.606597 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fec94-457c-4c09-a796-55cb409b775c service cinder] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:48.611799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fec94-457c-4c09-a796-55cb409b775c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:48.611799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135fec94-457c-4c09-a796-55cb409b775c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:48.611799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135fec94-457c-4c09-a796-55cb409b775c service cinder] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:48.612678 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 814/1638] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:48 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:48.613594 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 825/1639] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:12:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:12:48.614798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73d9e6-7e21-4dc9-a272-5eeace960414 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:48.614798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73d9e6-7e21-4dc9-a272-5eeace960414 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:48.614798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123304b2-1bb4-465d-b402-df41ed3e2753 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:48.614798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123304b2-1bb4-465d-b402-df41ed3e2753 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:48.616913 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123304b2-1bb4-465d-b402-df41ed3e2753 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:48.616913 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123304b2-1bb4-465d-b402-df41ed3e2753 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:48.616913 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123304b2-1bb4-465d-b402-df41ed3e2753 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:48.616913 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123304b2-1bb4-465d-b402-df41ed3e2753 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:48.616913 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 815/1640] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:12:48 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:12:48.710439 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:12:48.910951 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135fec94-457c-4c09-a796-55cb409b775c service cinder] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:48.922243 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135fec94-457c-4c09-a796-55cb409b775c service cinder] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:48.944330 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb140cd-e656-471a-a643-e8d88ade658c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:48.944988 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb140cd-e656-471a-a643-e8d88ade658c service cinder] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:48.945254 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb140cd-e656-471a-a643-e8d88ade658c service cinder] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:48.945560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb140cd-e656-471a-a643-e8d88ade658c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:48.982441 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb140cd-e656-471a-a643-e8d88ade658c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:48.983983 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb140cd-e656-471a-a643-e8d88ade658c service cinder] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:49.018002 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 826/1641] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:12:48 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:12:49.160511 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:12:49.251954 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09eae6-6905-4c70-83ac-2844ec220f9a None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:49.252465 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09eae6-6905-4c70-83ac-2844ec220f9a None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:49.252878 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09eae6-6905-4c70-83ac-2844ec220f9a None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:49.255250 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 816/1642] 10.222.0.40 () {54 vars in 962 bytes} [Wed Aug 3 05:12:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:12:49.270799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2a477-fe16-4b37-87b9-37f9a9802a2e None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:49.271979 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2a477-fe16-4b37-87b9-37f9a9802a2e None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:49.271979 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2a477-fe16-4b37-87b9-37f9a9802a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:49.377063 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-25c2a477-fe16-4b37-87b9-37f9a9802a2e None None] MFA Rules not processed for user `a3fa4a6c16274c18849869b55b33ac62`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:49.384239 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-25c2a477-fe16-4b37-87b9-37f9a9802a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:49.550049 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 827/1643] 10.222.0.40 () {58 vars in 1059 bytes} [Wed Aug 3 05:12:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:49.556643 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825a77b-263d-4886-9ae7-f3cb312a858f None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:49.557028 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825a77b-263d-4886-9ae7-f3cb312a858f None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:49.557382 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825a77b-263d-4886-9ae7-f3cb312a858f None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:49.558914 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 817/1644] 10.222.0.40 () {54 vars in 912 bytes} [Wed Aug 3 05:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 05:12:49.569404 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6325e-3b0e-4761-b39a-31cb4bcc8ab4 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:49.569734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6325e-3b0e-4761-b39a-31cb4bcc8ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:49.570042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6325e-3b0e-4761-b39a-31cb4bcc8ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:49.609461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-7ab6325e-3b0e-4761-b39a-31cb4bcc8ab4 None None] MFA Rules not processed for user `a3fa4a6c16274c18849869b55b33ac62`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:49.611095 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-7ab6325e-3b0e-4761-b39a-31cb4bcc8ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:49.616090 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9825a77b-263d-4886-9ae7-f3cb312a858f None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:49.634826 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 828/1645] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Aug 3 05:12:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:49.641203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9825a77b-263d-4886-9ae7-f3cb312a858f None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:49.645190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b3471-c434-49ee-9c9e-24fba9b952a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:49.645820 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b3471-c434-49ee-9c9e-24fba9b952a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:49.646109 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b3471-c434-49ee-9c9e-24fba9b952a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:49.648020 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b3471-c434-49ee-9c9e-24fba9b952a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:49.707088 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab6325e-3b0e-4761-b39a-31cb4bcc8ab4 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:49.731652 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab6325e-3b0e-4761-b39a-31cb4bcc8ab4 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:49.750106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5cb53a-4a68-4bd5-a36b-51e7da52e5ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:49.756878 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5cb53a-4a68-4bd5-a36b-51e7da52e5ec service cinder] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:49.757491 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5cb53a-4a68-4bd5-a36b-51e7da52e5ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:49.757862 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5cb53a-4a68-4bd5-a36b-51e7da52e5ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:49.806611 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5cb53a-4a68-4bd5-a36b-51e7da52e5ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:49.808143 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5cb53a-4a68-4bd5-a36b-51e7da52e5ec service cinder] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:49.824719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9b3471-c434-49ee-9c9e-24fba9b952a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:49.827558 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9b3471-c434-49ee-9c9e-24fba9b952a0 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:49.837290 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 829/1646] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:12:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:12:49.847201 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 818/1647] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:12:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:12:50.652088 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:12:51.798045 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5cb53a-4a68-4bd5-a36b-51e7da52e5ec service cinder] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:51.881854 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45ea20-6da3-4dff-ae93-776775e7424e None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:51.882466 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45ea20-6da3-4dff-ae93-776775e7424e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:51.882466 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45ea20-6da3-4dff-ae93-776775e7424e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:51.933296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5cb53a-4a68-4bd5-a36b-51e7da52e5ec service cinder] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:51.942819 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4de0d-f1f6-4921-a350-bfa55fabe84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:51.944412 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4de0d-f1f6-4921-a350-bfa55fabe84b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:51.944412 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4de0d-f1f6-4921-a350-bfa55fabe84b admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:51.944412 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4de0d-f1f6-4921-a350-bfa55fabe84b admin admin] PATH_INFO: `/v3/users/536e72a4b4c540df860ba3e2b3ee6081` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:51.945445 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb4de0d-f1f6-4921-a350-bfa55fabe84b admin admin] RBAC: Authorizing `identity:delete_user(user_id=536e72a4b4c540df860ba3e2b3ee6081)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:51.952596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb4de0d-f1f6-4921-a350-bfa55fabe84b admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:51.960941 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-9b45ea20-6da3-4dff-ae93-776775e7424e None None] MFA Rules not processed for user `bb0fe4cb98e0447cad3e126da1a8de9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:51.963481 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9b45ea20-6da3-4dff-ae93-776775e7424e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:52.008734 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:12:52.165168 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 819/1648] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:51 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:52.166821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50d3a8-db0a-4d3d-960d-8d1a81ed27c7 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:52.167375 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50d3a8-db0a-4d3d-960d-8d1a81ed27c7 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:52.167888 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50d3a8-db0a-4d3d-960d-8d1a81ed27c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:52.194233 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-dc50d3a8-db0a-4d3d-960d-8d1a81ed27c7 None None] MFA Rules not processed for user `bb0fe4cb98e0447cad3e126da1a8de9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:52.196509 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-dc50d3a8-db0a-4d3d-960d-8d1a81ed27c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:52.221361 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 820/1649] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:52.280905 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-2cb4de0d-f1f6-4921-a350-bfa55fabe84b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '536e72a4b4c540df860ba3e2b3ee6081'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:52.285114 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc97b6f-4e14-466e-ae79-319055d0505c None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:52.285114 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc97b6f-4e14-466e-ae79-319055d0505c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:52.285523 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc97b6f-4e14-466e-ae79-319055d0505c None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:52.287218 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 821/1650] 10.222.0.40 () {54 vars in 962 bytes} [Wed Aug 3 05:12:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:12:52.323282 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55bf39-911d-4d05-bfa1-2c410f801e31 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:52.323655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55bf39-911d-4d05-bfa1-2c410f801e31 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:52.324015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55bf39-911d-4d05-bfa1-2c410f801e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:52.349257 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-ba55bf39-911d-4d05-bfa1-2c410f801e31 None None] MFA Rules not processed for user `a3fa4a6c16274c18849869b55b33ac62`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:52.351419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ba55bf39-911d-4d05-bfa1-2c410f801e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:52.389046 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-2cb4de0d-f1f6-4921-a350-bfa55fabe84b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '536e72a4b4c540df860ba3e2b3ee6081'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:52.389694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-2cb4de0d-f1f6-4921-a350-bfa55fabe84b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '536e72a4b4c540df860ba3e2b3ee6081'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:52.390089 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 822/1651] 10.222.0.40 () {58 vars in 1059 bytes} [Wed Aug 3 05:12:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:52.408148 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55bf39-911d-4d05-bfa1-2c410f801e31 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:52.421748 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-2cb4de0d-f1f6-4921-a350-bfa55fabe84b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '536e72a4b4c540df860ba3e2b3ee6081'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:52.422663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-2cb4de0d-f1f6-4921-a350-bfa55fabe84b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '536e72a4b4c540df860ba3e2b3ee6081'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:52.435160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55bf39-911d-4d05-bfa1-2c410f801e31 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:52.451273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343d570-b2a4-4c5c-bbe9-a695cf9343ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:52.453646 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343d570-b2a4-4c5c-bbe9-a695cf9343ba service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:52.454175 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343d570-b2a4-4c5c-bbe9-a695cf9343ba service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:52.454556 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343d570-b2a4-4c5c-bbe9-a695cf9343ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:52.461515 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f343d570-b2a4-4c5c-bbe9-a695cf9343ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:52.462145 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 830/1652] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:12:51 2022] DELETE /identity/v3/users/536e72a4b4c540df860ba3e2b3ee6081 => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:12:52.465260 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f45e8c-0ceb-42a0-9b4b-f8c62d96a78c None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:52.467391 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f45e8c-0ceb-42a0-9b4b-f8c62d96a78c None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:52.467712 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f45e8c-0ceb-42a0-9b4b-f8c62d96a78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:52.487588 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f343d570-b2a4-4c5c-bbe9-a695cf9343ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:52.493489 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f343d570-b2a4-4c5c-bbe9-a695cf9343ba service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:52.521843 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-f1f45e8c-0ceb-42a0-9b4b-f8c62d96a78c None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:52.523837 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-f1f45e8c-0ceb-42a0-9b4b-f8c62d96a78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:52.532042 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 823/1653] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:12:52 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:12:52.721904 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 831/1654] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:52.817568 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343d570-b2a4-4c5c-bbe9-a695cf9343ba service nova] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:52.819113 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f343d570-b2a4-4c5c-bbe9-a695cf9343ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:52.945501 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343d570-b2a4-4c5c-bbe9-a695cf9343ba service nova] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:53.004019 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a9e9e-7698-4307-b165-b32293a4478b None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:53.004517 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a9e9e-7698-4307-b165-b32293a4478b None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:53.004609 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a9e9e-7698-4307-b165-b32293a4478b None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:53.007385 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 832/1655] 10.222.0.40 () {54 vars in 912 bytes} [Wed Aug 3 05:12:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:12:53.013883 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ae5b0-f62e-46ab-90e0-d1af99f6e8b1 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:53.020109 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ae5b0-f62e-46ab-90e0-d1af99f6e8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:53.020109 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ae5b0-f62e-46ab-90e0-d1af99f6e8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:53.049153 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-cd9ae5b0-f62e-46ab-90e0-d1af99f6e8b1 None None] MFA Rules not processed for user `a3fa4a6c16274c18849869b55b33ac62`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:53.050364 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-cd9ae5b0-f62e-46ab-90e0-d1af99f6e8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:53.060679 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db422bcf-3b23-4b91-bda1-a10b7a4072c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:53.061327 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db422bcf-3b23-4b91-bda1-a10b7a4072c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:53.061636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db422bcf-3b23-4b91-bda1-a10b7a4072c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:53.062279 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db422bcf-3b23-4b91-bda1-a10b7a4072c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:53.070711 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:12:53.084134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db422bcf-3b23-4b91-bda1-a10b7a4072c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:53.085643 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db422bcf-3b23-4b91-bda1-a10b7a4072c2 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:53.108756 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 824/1656] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:12:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:12:53.239720 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 833/1657] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Aug 3 05:12:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:53.259808 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db422bcf-3b23-4b91-bda1-a10b7a4072c2 service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:53.265841 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-db422bcf-3b23-4b91-bda1-a10b7a4072c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:53.428394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:12:53.455583 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ae5b0-f62e-46ab-90e0-d1af99f6e8b1 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:53.457297 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-cd9ae5b0-f62e-46ab-90e0-d1af99f6e8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:53.507105 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ae5b0-f62e-46ab-90e0-d1af99f6e8b1 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:53.521758 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db422bcf-3b23-4b91-bda1-a10b7a4072c2 service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:53.525790 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de223141-c6a8-4b96-853a-b9358fbbc075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:53.526821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de223141-c6a8-4b96-853a-b9358fbbc075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:53.527876 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de223141-c6a8-4b96-853a-b9358fbbc075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:53.528724 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d3dc0-5cbc-4d16-aff5-e9217a62d79c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cc14b33895443b893fab487bfe1858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:53.529487 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d3dc0-5cbc-4d16-aff5-e9217a62d79c service glance] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:53.529784 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d3dc0-5cbc-4d16-aff5-e9217a62d79c service glance] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:53.530066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d3dc0-5cbc-4d16-aff5-e9217a62d79c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:53.530281 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de223141-c6a8-4b96-853a-b9358fbbc075 admin admin] PATH_INFO: `/v3/users/3915a35da37a4e5cb8cfbf366f3e50a8` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:53.532088 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de223141-c6a8-4b96-853a-b9358fbbc075 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3915a35da37a4e5cb8cfbf366f3e50a8)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:53.540856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de223141-c6a8-4b96-853a-b9358fbbc075 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:53.559749 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d3dc0-5cbc-4d16-aff5-e9217a62d79c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:53.561402 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d3dc0-5cbc-4d16-aff5-e9217a62d79c service glance] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:53.611091 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 825/1658] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:12:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:12:53.938806 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-de223141-c6a8-4b96-853a-b9358fbbc075 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3915a35da37a4e5cb8cfbf366f3e50a8'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:54.034741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-de223141-c6a8-4b96-853a-b9358fbbc075 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3915a35da37a4e5cb8cfbf366f3e50a8'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:54.035400 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-de223141-c6a8-4b96-853a-b9358fbbc075 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3915a35da37a4e5cb8cfbf366f3e50a8'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:54.068745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-de223141-c6a8-4b96-853a-b9358fbbc075 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3915a35da37a4e5cb8cfbf366f3e50a8'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:54.069600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-de223141-c6a8-4b96-853a-b9358fbbc075 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3915a35da37a4e5cb8cfbf366f3e50a8'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:54.103993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 834/1659] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:12:53 2022] DELETE /identity/v3/users/3915a35da37a4e5cb8cfbf366f3e50a8 => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:12:54.105657 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30f25a-2ccf-400f-bee8-2342907a60fe None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:54.106108 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30f25a-2ccf-400f-bee8-2342907a60fe None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:54.106254 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30f25a-2ccf-400f-bee8-2342907a60fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:54.160540 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-2e30f25a-2ccf-400f-bee8-2342907a60fe None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:54.161354 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2e30f25a-2ccf-400f-bee8-2342907a60fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:54.330316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 826/1660] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:12:54.500186 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de223141-c6a8-4b96-853a-b9358fbbc075 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:54.501909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-de223141-c6a8-4b96-853a-b9358fbbc075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:54.711303 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de223141-c6a8-4b96-853a-b9358fbbc075 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:54.720952 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45142fec-cc34-4605-9fd5-c868d0b41ce9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:54.721554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45142fec-cc34-4605-9fd5-c868d0b41ce9 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:54.721840 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45142fec-cc34-4605-9fd5-c868d0b41ce9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:54.722114 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45142fec-cc34-4605-9fd5-c868d0b41ce9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:54.747712 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45142fec-cc34-4605-9fd5-c868d0b41ce9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:54.750205 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45142fec-cc34-4605-9fd5-c868d0b41ce9 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:54.777157 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 835/1661] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:12:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:12:55.962511 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e30f25a-2ccf-400f-bee8-2342907a60fe None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:55.962511 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2e30f25a-2ccf-400f-bee8-2342907a60fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:55.977425 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e916dc-0865-4a4c-92f8-819a095a58a3 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:55.978334 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e916dc-0865-4a4c-92f8-819a095a58a3 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:55.978758 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e916dc-0865-4a4c-92f8-819a095a58a3 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:55.980805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 836/1662] 10.222.0.40 () {54 vars in 912 bytes} [Wed Aug 3 05:12:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 05:12:55.986332 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e30f25a-2ccf-400f-bee8-2342907a60fe None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:55.997512 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c01cc3-0024-4f10-9670-5b6cfe2ece53 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:55.997986 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c01cc3-0024-4f10-9670-5b6cfe2ece53 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:55.998409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c01cc3-0024-4f10-9670-5b6cfe2ece53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:56.000438 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3c009c-56eb-4250-94a0-f78ea2fe16fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:56.001420 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c009c-56eb-4250-94a0-f78ea2fe16fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:56.001919 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c009c-56eb-4250-94a0-f78ea2fe16fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:56.002361 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c009c-56eb-4250-94a0-f78ea2fe16fb admin admin] PATH_INFO: `/v3/projects/078e30297f6446e0b4860ae523c2cf1f` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:56.005100 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3c009c-56eb-4250-94a0-f78ea2fe16fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=078e30297f6446e0b4860ae523c2cf1f)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:56.009548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3c009c-56eb-4250-94a0-f78ea2fe16fb admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:56.129071 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-10c01cc3-0024-4f10-9670-5b6cfe2ece53 None None] MFA Rules not processed for user `cff15ea4cbc64b068aa5f6586b5bff87`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:56.131043 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-10c01cc3-0024-4f10-9670-5b6cfe2ece53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:56.156833 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 837/1663] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Aug 3 05:12:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:12:56.196265 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5a3c009c-56eb-4250-94a0-f78ea2fe16fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '078e30297f6446e0b4860ae523c2cf1f', 'request_id': 'req-5a3c009c-56eb-4250-94a0-f78ea2fe16fb'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:56.325360 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5a3c009c-56eb-4250-94a0-f78ea2fe16fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '078e30297f6446e0b4860ae523c2cf1f', 'request_id': 'req-5a3c009c-56eb-4250-94a0-f78ea2fe16fb'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:56.333947 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5a3c009c-56eb-4250-94a0-f78ea2fe16fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '078e30297f6446e0b4860ae523c2cf1f', 'request_id': 'req-5a3c009c-56eb-4250-94a0-f78ea2fe16fb'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:56.365378 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5a3c009c-56eb-4250-94a0-f78ea2fe16fb admin admin] The token cache is being invalidate because project 078e30297f6446e0b4860ae523c2cf1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:12:56.365566 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5a3c009c-56eb-4250-94a0-f78ea2fe16fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:56.369586 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5a3c009c-56eb-4250-94a0-f78ea2fe16fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:56.372571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 827/1664] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:12:55 2022] DELETE /identity/v3/projects/078e30297f6446e0b4860ae523c2cf1f => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:12:56.378430 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c01cc3-0024-4f10-9670-5b6cfe2ece53 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:56.387080 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3c009c-56eb-4250-94a0-f78ea2fe16fb admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:56.389299 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-5a3c009c-56eb-4250-94a0-f78ea2fe16fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:56.437854 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:12:56.447683 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c01cc3-0024-4f10-9670-5b6cfe2ece53 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:56.531450 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3c009c-56eb-4250-94a0-f78ea2fe16fb admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:56.542698 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f19a7-b587-455a-86c6-fe58682d4885 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:56.545447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f19a7-b587-455a-86c6-fe58682d4885 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:56.545936 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f19a7-b587-455a-86c6-fe58682d4885 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:56.546217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f19a7-b587-455a-86c6-fe58682d4885 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:56.553184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-572f19a7-b587-455a-86c6-fe58682d4885 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:56.578887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f19a7-b587-455a-86c6-fe58682d4885 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:56.580856 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f19a7-b587-455a-86c6-fe58682d4885 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:56.597566 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898ebe4-2b7f-456f-82ae-1236e73442fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:56.605042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898ebe4-2b7f-456f-82ae-1236e73442fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:56.605042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898ebe4-2b7f-456f-82ae-1236e73442fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:56.605243 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:12:56.605327 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898ebe4-2b7f-456f-82ae-1236e73442fb admin admin] PATH_INFO: `/v3/users/0b262d10633b415b8ef593e434be58f0` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:56.607661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c898ebe4-2b7f-456f-82ae-1236e73442fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b262d10633b415b8ef593e434be58f0)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:56.611050 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 828/1665] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:12:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:12:56.614963 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c898ebe4-2b7f-456f-82ae-1236e73442fb admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:56.824389 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-c898ebe4-2b7f-456f-82ae-1236e73442fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b262d10633b415b8ef593e434be58f0'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:56.937014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-c898ebe4-2b7f-456f-82ae-1236e73442fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b262d10633b415b8ef593e434be58f0'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:56.937817 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-c898ebe4-2b7f-456f-82ae-1236e73442fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b262d10633b415b8ef593e434be58f0'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:56.959613 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-c898ebe4-2b7f-456f-82ae-1236e73442fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b262d10633b415b8ef593e434be58f0'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:56.960699 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-c898ebe4-2b7f-456f-82ae-1236e73442fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b262d10633b415b8ef593e434be58f0'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:56.991423 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 838/1666] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:12:56 2022] DELETE /identity/v3/users/0b262d10633b415b8ef593e434be58f0 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:12:57.516161 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f19a7-b587-455a-86c6-fe58682d4885 service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:57.522470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-572f19a7-b587-455a-86c6-fe58682d4885 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:57.551487 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f19a7-b587-455a-86c6-fe58682d4885 service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:57.713065 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359433a0-10eb-4ac4-b116-23573b53a83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:57.715043 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359433a0-10eb-4ac4-b116-23573b53a83f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:57.716280 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359433a0-10eb-4ac4-b116-23573b53a83f admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:57.716713 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359433a0-10eb-4ac4-b116-23573b53a83f admin admin] PATH_INFO: `/v3/projects/cdd26a4536e04491972ff6618289fc8f` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:57.719849 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359433a0-10eb-4ac4-b116-23573b53a83f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdd26a4536e04491972ff6618289fc8f)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:57.727359 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359433a0-10eb-4ac4-b116-23573b53a83f admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:57.945018 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-359433a0-10eb-4ac4-b116-23573b53a83f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdd26a4536e04491972ff6618289fc8f', 'request_id': 'req-359433a0-10eb-4ac4-b116-23573b53a83f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:58.064755 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-359433a0-10eb-4ac4-b116-23573b53a83f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdd26a4536e04491972ff6618289fc8f', 'request_id': 'req-359433a0-10eb-4ac4-b116-23573b53a83f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:58.090313 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-359433a0-10eb-4ac4-b116-23573b53a83f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdd26a4536e04491972ff6618289fc8f', 'request_id': 'req-359433a0-10eb-4ac4-b116-23573b53a83f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:58.131694 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-359433a0-10eb-4ac4-b116-23573b53a83f admin admin] The token cache is being invalidate because project cdd26a4536e04491972ff6618289fc8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:12:58.131991 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-359433a0-10eb-4ac4-b116-23573b53a83f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:58.133051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-359433a0-10eb-4ac4-b116-23573b53a83f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:58.137428 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 829/1667] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:12:57 2022] DELETE /identity/v3/projects/cdd26a4536e04491972ff6618289fc8f => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:12:58.141360 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898ebe4-2b7f-456f-82ae-1236e73442fb admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:58.143290 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-c898ebe4-2b7f-456f-82ae-1236e73442fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:58.163929 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52fc81-a7f4-4707-b447-38a522756b9c None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:58.164217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52fc81-a7f4-4707-b447-38a522756b9c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:58.164496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52fc81-a7f4-4707-b447-38a522756b9c None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:58.167207 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 830/1668] 10.222.0.40 () {54 vars in 961 bytes} [Wed Aug 3 05:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 05:12:58.173915 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898ebe4-2b7f-456f-82ae-1236e73442fb admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:58.177693 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b7064-740b-44ea-bf69-143322a3a707 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:58.177990 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b7064-740b-44ea-bf69-143322a3a707 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:58.178272 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b7064-740b-44ea-bf69-143322a3a707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:58.245346 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-f62b7064-740b-44ea-bf69-143322a3a707 None None] MFA Rules not processed for user `59846d88f4314c2382aa5cb8bae5053b`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:58.250728 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f62b7064-740b-44ea-bf69-143322a3a707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:58.343559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b831ba-628b-412c-bbe4-32b2a7549dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:58.343559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b831ba-628b-412c-bbe4-32b2a7549dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:58.343559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b831ba-628b-412c-bbe4-32b2a7549dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:58.343559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b831ba-628b-412c-bbe4-32b2a7549dd7 admin admin] PATH_INFO: `/v3/users/352bcbfb3ffa4ba8b5109abb9adf45c4` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:58.343912 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b831ba-628b-412c-bbe4-32b2a7549dd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=352bcbfb3ffa4ba8b5109abb9adf45c4)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:58.343912 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b831ba-628b-412c-bbe4-32b2a7549dd7 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:58.393054 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:12:58.401784 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 831/1669] 10.222.0.40 () {58 vars in 1058 bytes} [Wed Aug 3 05:12:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:58.402192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b7064-740b-44ea-bf69-143322a3a707 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:58.403953 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f62b7064-740b-44ea-bf69-143322a3a707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:58.420172 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b7064-740b-44ea-bf69-143322a3a707 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:58.424933 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb43ac-5541-431e-b3bd-23025d713d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:58.425547 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb43ac-5541-431e-b3bd-23025d713d53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:58.425840 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb43ac-5541-431e-b3bd-23025d713d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:58.426107 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb43ac-5541-431e-b3bd-23025d713d53 admin admin] PATH_INFO: `/v3/projects/346d41c01e174b19875cd4c6f14cca0c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:58.427927 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb43ac-5541-431e-b3bd-23025d713d53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=346d41c01e174b19875cd4c6f14cca0c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:58.429914 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb43ac-5541-431e-b3bd-23025d713d53 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:58.456109 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:12:58.603471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-35fb43ac-5541-431e-b3bd-23025d713d53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '346d41c01e174b19875cd4c6f14cca0c', 'request_id': 'req-35fb43ac-5541-431e-b3bd-23025d713d53'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:58.624758 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-88b831ba-628b-412c-bbe4-32b2a7549dd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '352bcbfb3ffa4ba8b5109abb9adf45c4'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:58.702661 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-35fb43ac-5541-431e-b3bd-23025d713d53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '346d41c01e174b19875cd4c6f14cca0c', 'request_id': 'req-35fb43ac-5541-431e-b3bd-23025d713d53'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:58.712771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-35fb43ac-5541-431e-b3bd-23025d713d53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '346d41c01e174b19875cd4c6f14cca0c', 'request_id': 'req-35fb43ac-5541-431e-b3bd-23025d713d53'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:58.736612 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-35fb43ac-5541-431e-b3bd-23025d713d53 admin admin] The token cache is being invalidate because project 346d41c01e174b19875cd4c6f14cca0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:12:58.736612 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-35fb43ac-5541-431e-b3bd-23025d713d53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:58.737121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-35fb43ac-5541-431e-b3bd-23025d713d53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:58.747121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 832/1670] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:12:58 2022] DELETE /identity/v3/projects/346d41c01e174b19875cd4c6f14cca0c => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:12:58.747121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167f868-78a5-4dec-a5e9-415e30168e10 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:58.747121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167f868-78a5-4dec-a5e9-415e30168e10 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:58.747121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167f868-78a5-4dec-a5e9-415e30168e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:58.758225 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-88b831ba-628b-412c-bbe4-32b2a7549dd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '352bcbfb3ffa4ba8b5109abb9adf45c4'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:58.759028 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-88b831ba-628b-412c-bbe4-32b2a7549dd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '352bcbfb3ffa4ba8b5109abb9adf45c4'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:58.804383 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-88b831ba-628b-412c-bbe4-32b2a7549dd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '352bcbfb3ffa4ba8b5109abb9adf45c4'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:58.805155 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-88b831ba-628b-412c-bbe4-32b2a7549dd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '352bcbfb3ffa4ba8b5109abb9adf45c4'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:12:58.813661 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-3167f868-78a5-4dec-a5e9-415e30168e10 None None] MFA Rules not processed for user `bb0fe4cb98e0447cad3e126da1a8de9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:58.816853 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3167f868-78a5-4dec-a5e9-415e30168e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:58.839965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 839/1671] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:12:58 2022] DELETE /identity/v3/users/352bcbfb3ffa4ba8b5109abb9adf45c4 => generated 0 bytes in 699 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:12:58.853663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b831ba-628b-412c-bbe4-32b2a7549dd7 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:58.859344 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-88b831ba-628b-412c-bbe4-32b2a7549dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:59.072420 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 833/1672] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:59.075286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e6288-6288-44d9-9d6b-ec8ce7086f7b None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:59.075970 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e6288-6288-44d9-9d6b-ec8ce7086f7b None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:59.076251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e6288-6288-44d9-9d6b-ec8ce7086f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:59.141922 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-ca3e6288-6288-44d9-9d6b-ec8ce7086f7b None None] MFA Rules not processed for user `747a608061504d7693f0f99ecdfa476e`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:59.143167 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ca3e6288-6288-44d9-9d6b-ec8ce7086f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:59.152513 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b831ba-628b-412c-bbe4-32b2a7549dd7 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:59.168285 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a76816-13dd-4105-9619-6a7417575d00 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59846d88f4314c2382aa5cb8bae5053b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:59.171866 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a76816-13dd-4105-9619-6a7417575d00 service placement] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:59.172287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a76816-13dd-4105-9619-6a7417575d00 service placement] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:59.173537 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a76816-13dd-4105-9619-6a7417575d00 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:59.193173 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-37a76816-13dd-4105-9619-6a7417575d00 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:59.220795 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a76816-13dd-4105-9619-6a7417575d00 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:59.222438 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a76816-13dd-4105-9619-6a7417575d00 service placement] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:59.239958 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 840/1673] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:12:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:12:59.241304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4911de-2ce4-4820-97d3-a08b379bac08 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:59.241867 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4911de-2ce4-4820-97d3-a08b379bac08 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:59.242350 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4911de-2ce4-4820-97d3-a08b379bac08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:59.266596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-1e4911de-2ce4-4820-97d3-a08b379bac08 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:59.275994 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-1e4911de-2ce4-4820-97d3-a08b379bac08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:59.384011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 834/1674] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:59 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:59.395173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e6288-6288-44d9-9d6b-ec8ce7086f7b None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:59.397469 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ca3e6288-6288-44d9-9d6b-ec8ce7086f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:59.426593 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 841/1675] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:12:59.436579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333efe7-d18a-4c9b-95ce-3409919a2552 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:59.437103 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333efe7-d18a-4c9b-95ce-3409919a2552 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:59.437527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333efe7-d18a-4c9b-95ce-3409919a2552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:59.481667 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-f333efe7-d18a-4c9b-95ce-3409919a2552 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:12:59.484668 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-f333efe7-d18a-4c9b-95ce-3409919a2552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:59.508334 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 842/1676] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:12:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:12:59.509659 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333efe7-d18a-4c9b-95ce-3409919a2552 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:59.563407 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333efe7-d18a-4c9b-95ce-3409919a2552 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:59.569935 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11696cfd-fbb4-4418-bb81-3939520c3653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:59.571682 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11696cfd-fbb4-4418-bb81-3939520c3653 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:59.575826 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11696cfd-fbb4-4418-bb81-3939520c3653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:59.577085 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11696cfd-fbb4-4418-bb81-3939520c3653 admin admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:59.578060 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11696cfd-fbb4-4418-bb81-3939520c3653 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:12:59.578669 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11696cfd-fbb4-4418-bb81-3939520c3653 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:59.582962 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11696cfd-fbb4-4418-bb81-3939520c3653 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:59.606406 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 843/1677] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:12:59.612066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11696cfd-fbb4-4418-bb81-3939520c3653 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:59.636220 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11696cfd-fbb4-4418-bb81-3939520c3653 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:59.647562 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d703f1-bf21-4b7b-bc2b-22816e7b6c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:59.648522 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d703f1-bf21-4b7b-bc2b-22816e7b6c61 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:59.649362 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d703f1-bf21-4b7b-bc2b-22816e7b6c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:59.649840 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d703f1-bf21-4b7b-bc2b-22816e7b6c61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:59.650867 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d703f1-bf21-4b7b-bc2b-22816e7b6c61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:59.653403 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d703f1-bf21-4b7b-bc2b-22816e7b6c61 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:59.686106 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e6288-6288-44d9-9d6b-ec8ce7086f7b None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:59.705782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b796c8fc-7abf-4547-b5a4-5265677f73da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:59.706437 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796c8fc-7abf-4547-b5a4-5265677f73da service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:59.706720 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796c8fc-7abf-4547-b5a4-5265677f73da service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:59.707065 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796c8fc-7abf-4547-b5a4-5265677f73da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:59.746372 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b796c8fc-7abf-4547-b5a4-5265677f73da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:59.748248 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b796c8fc-7abf-4547-b5a4-5265677f73da service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:12:59.776878 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 844/1678] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:12:59 2022] POST /identity/v3/projects => generated 363 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:12:59.779439 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 835/1679] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:12:59 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:12:59.808805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d703f1-bf21-4b7b-bc2b-22816e7b6c61 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:59.809005 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b796c8fc-7abf-4547-b5a4-5265677f73da service nova] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:12:59.810651 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b796c8fc-7abf-4547-b5a4-5265677f73da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:12:59.833191 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b796c8fc-7abf-4547-b5a4-5265677f73da service nova] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:12:59.932674 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488334d-9c2c-4dce-9123-d604f0620f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:12:59.935436 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488334d-9c2c-4dce-9123-d604f0620f19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:12:59.935769 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488334d-9c2c-4dce-9123-d604f0620f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:12:59.936047 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488334d-9c2c-4dce-9123-d604f0620f19 admin admin] PATH_INFO: `/v3/projects/ebbe2fc1c4b34d4e9324555109c39c57` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:12:59.938884 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3488334d-9c2c-4dce-9123-d604f0620f19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebbe2fc1c4b34d4e9324555109c39c57)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:12:59.945550 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3488334d-9c2c-4dce-9123-d604f0620f19 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:00.029052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:00.051714 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d703f1-bf21-4b7b-bc2b-22816e7b6c61 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:00.057590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9787af-6762-439c-b4ea-45d3f9b88a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:00.058676 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9787af-6762-439c-b4ea-45d3f9b88a76 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:00.059230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9787af-6762-439c-b4ea-45d3f9b88a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:00.059698 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9787af-6762-439c-b4ea-45d3f9b88a76 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:00.061652 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9787af-6762-439c-b4ea-45d3f9b88a76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:00.066876 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9787af-6762-439c-b4ea-45d3f9b88a76 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:00.159875 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-3488334d-9c2c-4dce-9123-d604f0620f19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebbe2fc1c4b34d4e9324555109c39c57', 'request_id': 'req-3488334d-9c2c-4dce-9123-d604f0620f19'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:00.264364 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 845/1680] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:12:59 2022] POST /identity/v3/users => generated 282 bytes in 456 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:00.268928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9787af-6762-439c-b4ea-45d3f9b88a76 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:00.287419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-3488334d-9c2c-4dce-9123-d604f0620f19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebbe2fc1c4b34d4e9324555109c39c57', 'request_id': 'req-3488334d-9c2c-4dce-9123-d604f0620f19'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:00.301937 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:00.310193 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9787af-6762-439c-b4ea-45d3f9b88a76 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:00.323134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-3488334d-9c2c-4dce-9123-d604f0620f19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebbe2fc1c4b34d4e9324555109c39c57', 'request_id': 'req-3488334d-9c2c-4dce-9123-d604f0620f19'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:00.384547 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-3488334d-9c2c-4dce-9123-d604f0620f19 admin admin] The token cache is being invalidate because project ebbe2fc1c4b34d4e9324555109c39c57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:13:00.384654 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-3488334d-9c2c-4dce-9123-d604f0620f19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:00.393420 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-3488334d-9c2c-4dce-9123-d604f0620f19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:00.404556 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 836/1681] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:12:59 2022] DELETE /identity/v3/projects/ebbe2fc1c4b34d4e9324555109c39c57 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:00.428934 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec10e4a-3375-4881-bf43-404462ee82b9 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:00.428934 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec10e4a-3375-4881-bf43-404462ee82b9 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:00.428934 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec10e4a-3375-4881-bf43-404462ee82b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:00.429479 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4320e8-fc61-4f1e-aaf4-82386d845e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:00.429479 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4320e8-fc61-4f1e-aaf4-82386d845e95 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:00.429479 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4320e8-fc61-4f1e-aaf4-82386d845e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:00.429479 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4320e8-fc61-4f1e-aaf4-82386d845e95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:00.429816 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4320e8-fc61-4f1e-aaf4-82386d845e95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:00.439223 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4320e8-fc61-4f1e-aaf4-82386d845e95 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:00.468008 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 846/1682] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:00.470635 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4320e8-fc61-4f1e-aaf4-82386d845e95 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:00.475841 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-5d4320e8-fc61-4f1e-aaf4-82386d845e95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:00.497612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4320e8-fc61-4f1e-aaf4-82386d845e95 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:00.504844 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028b005-7391-401a-a5e7-16fc3d6cd12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:00.506767 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028b005-7391-401a-a5e7-16fc3d6cd12f admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:00.507331 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028b005-7391-401a-a5e7-16fc3d6cd12f admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:00.507806 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028b005-7391-401a-a5e7-16fc3d6cd12f admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:00.508740 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-2ec10e4a-3375-4881-bf43-404462ee82b9 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:00.509006 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4028b005-7391-401a-a5e7-16fc3d6cd12f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:00.510021 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2ec10e4a-3375-4881-bf43-404462ee82b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:00.511019 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4028b005-7391-401a-a5e7-16fc3d6cd12f admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:00.536091 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 837/1683] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:00.538426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7cbca-b0ea-45ce-883e-166cac311db0 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:00.538798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7cbca-b0ea-45ce-883e-166cac311db0 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:00.540234 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7cbca-b0ea-45ce-883e-166cac311db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:00.546627 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 847/1684] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:00.550659 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028b005-7391-401a-a5e7-16fc3d6cd12f admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:00.573949 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028b005-7391-401a-a5e7-16fc3d6cd12f admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:00.580829 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-dde7cbca-b0ea-45ce-883e-166cac311db0 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:00.582975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-dde7cbca-b0ea-45ce-883e-166cac311db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:00.587475 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca5c29c-4a7b-444b-816c-aeccf6c8c198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:00.588751 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5c29c-4a7b-444b-816c-aeccf6c8c198 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:00.589250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5c29c-4a7b-444b-816c-aeccf6c8c198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:00.589739 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5c29c-4a7b-444b-816c-aeccf6c8c198 admin admin] PATH_INFO: `/v3/projects/28d20d96eda048d888fa884535e48a0c/users/b967b5aa62254fc0b086344bccb041b8/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:00.617228 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 838/1685] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:00.624528 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7cbca-b0ea-45ce-883e-166cac311db0 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:00.655846 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7cbca-b0ea-45ce-883e-166cac311db0 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:00.656944 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca5c29c-4a7b-444b-816c-aeccf6c8c198 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28d20d96eda048d888fa884535e48a0c, user_id=b967b5aa62254fc0b086344bccb041b8, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:00.690079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca5c29c-4a7b-444b-816c-aeccf6c8c198 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:00.702672 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1b365-96b4-4f19-8855-b21f7f59e8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:00.706612 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1b365-96b4-4f19-8855-b21f7f59e8d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:00.706977 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1b365-96b4-4f19-8855-b21f7f59e8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:00.707267 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1b365-96b4-4f19-8855-b21f7f59e8d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:00.707994 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f1b365-96b4-4f19-8855-b21f7f59e8d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:13:00.708874 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f1b365-96b4-4f19-8855-b21f7f59e8d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:00.710494 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f1b365-96b4-4f19-8855-b21f7f59e8d1 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:00.730946 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 839/1686] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:13:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:13:00.735682 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1b365-96b4-4f19-8855-b21f7f59e8d1 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:00.753912 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1b365-96b4-4f19-8855-b21f7f59e8d1 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:00.755960 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f6ce9-8578-4b00-bc6a-df590b76eb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:00.756748 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f6ce9-8578-4b00-bc6a-df590b76eb51 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:00.757209 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f6ce9-8578-4b00-bc6a-df590b76eb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:00.757586 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f6ce9-8578-4b00-bc6a-df590b76eb51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:00.758423 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f6ce9-8578-4b00-bc6a-df590b76eb51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:00.760744 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f6ce9-8578-4b00-bc6a-df590b76eb51 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:00.867812 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 848/1687] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:00 2022] PUT /identity/v3/projects/28d20d96eda048d888fa884535e48a0c/users/b967b5aa62254fc0b086344bccb041b8/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:00.870041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85050303-ca9d-4dd3-8132-a0acf49212dc None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:00.870331 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85050303-ca9d-4dd3-8132-a0acf49212dc None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:00.870611 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85050303-ca9d-4dd3-8132-a0acf49212dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:00.884978 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 840/1688] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:13:00 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:13:00.891180 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f6ce9-8578-4b00-bc6a-df590b76eb51 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:00.962191 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-85050303-ca9d-4dd3-8132-a0acf49212dc None None] MFA Rules not processed for user `b967b5aa62254fc0b086344bccb041b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:00.963544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-85050303-ca9d-4dd3-8132-a0acf49212dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:01.041515 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f6ce9-8578-4b00-bc6a-df590b76eb51 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:01.056536 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782902d6-c507-4c93-9c5b-05117162f3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:01.059477 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782902d6-c507-4c93-9c5b-05117162f3f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:01.060282 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782902d6-c507-4c93-9c5b-05117162f3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:01.061349 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782902d6-c507-4c93-9c5b-05117162f3f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:01.063486 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782902d6-c507-4c93-9c5b-05117162f3f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:01.069472 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782902d6-c507-4c93-9c5b-05117162f3f3 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:01.131578 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 849/1689] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:01.155400 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85050303-ca9d-4dd3-8132-a0acf49212dc None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:01.203392 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 841/1690] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:13:00 2022] POST /identity/v3/users => generated 293 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:01.207737 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782902d6-c507-4c93-9c5b-05117162f3f3 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:01.214545 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85050303-ca9d-4dd3-8132-a0acf49212dc None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:01.236759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5afdc7-d5f6-48ae-bc2d-2e9f65e83dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:01.237416 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5afdc7-d5f6-48ae-bc2d-2e9f65e83dd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:01.237976 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5afdc7-d5f6-48ae-bc2d-2e9f65e83dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:01.237976 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5afdc7-d5f6-48ae-bc2d-2e9f65e83dd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:01.238657 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5afdc7-d5f6-48ae-bc2d-2e9f65e83dd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:01.240637 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782902d6-c507-4c93-9c5b-05117162f3f3 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:01.252822 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ecd200-010a-4046-9745-46a912df6677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:01.254425 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ecd200-010a-4046-9745-46a912df6677 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:01.256166 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ecd200-010a-4046-9745-46a912df6677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:01.256651 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ecd200-010a-4046-9745-46a912df6677 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:01.257779 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5afdc7-d5f6-48ae-bc2d-2e9f65e83dd4 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:01.259416 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ecd200-010a-4046-9745-46a912df6677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:01.261870 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ecd200-010a-4046-9745-46a912df6677 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:01.304783 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 842/1691] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:01.308923 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ecd200-010a-4046-9745-46a912df6677 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:01.343724 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ecd200-010a-4046-9745-46a912df6677 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:01.362396 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e0d141-caab-4466-82a8-f43fad1976df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:01.365577 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0d141-caab-4466-82a8-f43fad1976df admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:01.366226 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0d141-caab-4466-82a8-f43fad1976df admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:01.366695 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0d141-caab-4466-82a8-f43fad1976df admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:01.370259 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e0d141-caab-4466-82a8-f43fad1976df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:01.387036 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e0d141-caab-4466-82a8-f43fad1976df admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:01.430509 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 843/1692] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:01.439400 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e0d141-caab-4466-82a8-f43fad1976df admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:01.449389 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 850/1693] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:13:01 2022] POST /identity/v3/projects => generated 365 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:01.457389 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5afdc7-d5f6-48ae-bc2d-2e9f65e83dd4 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:01.462269 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e0d141-caab-4466-82a8-f43fad1976df admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:01.668272 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803c31cb-d408-476f-9e1a-b4df0a449e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:01.670618 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803c31cb-d408-476f-9e1a-b4df0a449e32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:01.671185 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803c31cb-d408-476f-9e1a-b4df0a449e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:01.671472 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803c31cb-d408-476f-9e1a-b4df0a449e32 admin admin] PATH_INFO: `/v3/projects/15d57423afdc4b6c8b3a2e9d3e6d5328/users/ae5fa3f9e36940d3be189260adfe3321/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:01.674640 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5afdc7-d5f6-48ae-bc2d-2e9f65e83dd4 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:01.687547 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085a4f1-9bc3-4b66-a090-15cd70b89948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:01.688721 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085a4f1-9bc3-4b66-a090-15cd70b89948 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:01.689147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085a4f1-9bc3-4b66-a090-15cd70b89948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:01.689548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085a4f1-9bc3-4b66-a090-15cd70b89948 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:01.690455 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085a4f1-9bc3-4b66-a090-15cd70b89948 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:01.693504 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085a4f1-9bc3-4b66-a090-15cd70b89948 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:01.731468 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803c31cb-d408-476f-9e1a-b4df0a449e32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15d57423afdc4b6c8b3a2e9d3e6d5328, user_id=ae5fa3f9e36940d3be189260adfe3321, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:01.734775 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803c31cb-d408-476f-9e1a-b4df0a449e32 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:01.848012 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 851/1694] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:13:01 2022] POST /identity/v3/users => generated 289 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:01.851335 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 844/1695] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:01 2022] PUT /identity/v3/projects/15d57423afdc4b6c8b3a2e9d3e6d5328/users/ae5fa3f9e36940d3be189260adfe3321/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:01.852520 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085a4f1-9bc3-4b66-a090-15cd70b89948 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:01.859705 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd8d7d-c3b0-40da-99e0-16c5ff3d8796 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:01.860101 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd8d7d-c3b0-40da-99e0-16c5ff3d8796 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:01.860401 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd8d7d-c3b0-40da-99e0-16c5ff3d8796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:01.896961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085a4f1-9bc3-4b66-a090-15cd70b89948 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:01.979822 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-78bd8d7d-c3b0-40da-99e0-16c5ff3d8796 None None] MFA Rules not processed for user `ae5fa3f9e36940d3be189260adfe3321`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:01.983124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-78bd8d7d-c3b0-40da-99e0-16c5ff3d8796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:02.193666 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe98ae2-13a3-438f-8139-9bf7f707df71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:02.217018 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe98ae2-13a3-438f-8139-9bf7f707df71 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:02.219865 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe98ae2-13a3-438f-8139-9bf7f707df71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:02.222039 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe98ae2-13a3-438f-8139-9bf7f707df71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:02.222039 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe98ae2-13a3-438f-8139-9bf7f707df71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:02.230782 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe98ae2-13a3-438f-8139-9bf7f707df71 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:02.261731 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 852/1696] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:02.271769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe98ae2-13a3-438f-8139-9bf7f707df71 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:02.288051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 845/1697] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 429 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:02.290645 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe98ae2-13a3-438f-8139-9bf7f707df71 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:02.292229 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd8d7d-c3b0-40da-99e0-16c5ff3d8796 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:02.308492 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a215ac04-9f90-44d7-a176-e99e6c0b746c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:02.310923 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215ac04-9f90-44d7-a176-e99e6c0b746c admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:02.312161 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215ac04-9f90-44d7-a176-e99e6c0b746c admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:02.312765 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd8d7d-c3b0-40da-99e0-16c5ff3d8796 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:02.313301 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215ac04-9f90-44d7-a176-e99e6c0b746c admin admin] PATH_INFO: `/v3/projects/23c74638205446c888bd8273c25bb103/users/8da1bae26adb4d2c9ce009ae19f96639/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:02.327140 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a74ec1-dfa8-4e4c-8370-69a064674dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:02.338760 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a74ec1-dfa8-4e4c-8370-69a064674dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:02.341742 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a74ec1-dfa8-4e4c-8370-69a064674dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:02.352654 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a74ec1-dfa8-4e4c-8370-69a064674dc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:02.352786 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a215ac04-9f90-44d7-a176-e99e6c0b746c admin admin] RBAC: Authorizing `identity:create_grant(project_id=23c74638205446c888bd8273c25bb103, user_id=8da1bae26adb4d2c9ce009ae19f96639, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:02.352864 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a74ec1-dfa8-4e4c-8370-69a064674dc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:02.352947 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a215ac04-9f90-44d7-a176-e99e6c0b746c admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:02.353022 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a74ec1-dfa8-4e4c-8370-69a064674dc0 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:02.469033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 853/1698] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:02 2022] PUT /identity/v3/projects/23c74638205446c888bd8273c25bb103/users/8da1bae26adb4d2c9ce009ae19f96639/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:02.472604 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a215ac04-9f90-44d7-a176-e99e6c0b746c admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:02.511689 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 846/1699] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:13:02 2022] POST /identity/v3/projects => generated 387 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:13:02.514101 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a74ec1-dfa8-4e4c-8370-69a064674dc0 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:02.690435 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a215ac04-9f90-44d7-a176-e99e6c0b746c admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:02.760535 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a74ec1-dfa8-4e4c-8370-69a064674dc0 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:02.796809 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d86a1-5d3d-4c9d-ad13-07c55a418889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:02.798162 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d86a1-5d3d-4c9d-ad13-07c55a418889 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:02.798654 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d86a1-5d3d-4c9d-ad13-07c55a418889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:02.799163 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d86a1-5d3d-4c9d-ad13-07c55a418889 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:02.800247 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d86a1-5d3d-4c9d-ad13-07c55a418889 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:02.803186 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d86a1-5d3d-4c9d-ad13-07c55a418889 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:02.847222 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04dd04-9f64-4b27-b77b-43ad1c636fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:02.847816 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04dd04-9f64-4b27-b77b-43ad1c636fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:02.849134 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04dd04-9f64-4b27-b77b-43ad1c636fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:02.850861 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04dd04-9f64-4b27-b77b-43ad1c636fb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:02.851905 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04dd04-9f64-4b27-b77b-43ad1c636fb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:02.852820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:02.857143 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04dd04-9f64-4b27-b77b-43ad1c636fb7 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:02.876100 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 854/1700] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:02.878110 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04dd04-9f64-4b27-b77b-43ad1c636fb7 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:02.892239 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04dd04-9f64-4b27-b77b-43ad1c636fb7 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:02.920064 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575558a-7289-4f6a-98b6-d9b9a1bddcd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:02.921137 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575558a-7289-4f6a-98b6-d9b9a1bddcd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:02.923236 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575558a-7289-4f6a-98b6-d9b9a1bddcd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:02.934791 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 847/1701] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:13:02 2022] POST /identity/v3/users => generated 300 bytes in 421 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:13:02.935198 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575558a-7289-4f6a-98b6-d9b9a1bddcd1 admin admin] PATH_INFO: `/v3/projects/23c74638205446c888bd8273c25bb103/users/8da1bae26adb4d2c9ce009ae19f96639/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:02.940023 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d86a1-5d3d-4c9d-ad13-07c55a418889 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:02.944959 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575558a-7289-4f6a-98b6-d9b9a1bddcd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23c74638205446c888bd8273c25bb103, user_id=8da1bae26adb4d2c9ce009ae19f96639, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:02.953935 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575558a-7289-4f6a-98b6-d9b9a1bddcd1 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:03.019136 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d86a1-5d3d-4c9d-ad13-07c55a418889 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:03.054664 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 855/1702] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:02 2022] PUT /identity/v3/projects/23c74638205446c888bd8273c25bb103/users/8da1bae26adb4d2c9ce009ae19f96639/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:03.059142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688215b-78ea-4a01-a507-4f6d402dc28e None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:03.059697 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688215b-78ea-4a01-a507-4f6d402dc28e None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:03.059801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688215b-78ea-4a01-a507-4f6d402dc28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:03.080761 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:03.212498 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-1688215b-78ea-4a01-a507-4f6d402dc28e None None] MFA Rules not processed for user `8da1bae26adb4d2c9ce009ae19f96639`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:03.228154 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-1688215b-78ea-4a01-a507-4f6d402dc28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:03.329879 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1dea93-6d90-4654-b7af-8304355f6edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:03.330673 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1dea93-6d90-4654-b7af-8304355f6edf admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:03.332578 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1dea93-6d90-4654-b7af-8304355f6edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:03.332879 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1dea93-6d90-4654-b7af-8304355f6edf admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:03.333678 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1dea93-6d90-4654-b7af-8304355f6edf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:03.342168 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1dea93-6d90-4654-b7af-8304355f6edf admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:03.382289 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 848/1703] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 440 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:03.383381 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1dea93-6d90-4654-b7af-8304355f6edf admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:03.388788 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-5a1dea93-6d90-4654-b7af-8304355f6edf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:03.419721 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1dea93-6d90-4654-b7af-8304355f6edf admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:03.487530 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f234fec0-5c56-436c-bfa6-b0147d58a047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:03.493443 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234fec0-5c56-436c-bfa6-b0147d58a047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:03.495013 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234fec0-5c56-436c-bfa6-b0147d58a047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:03.495948 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234fec0-5c56-436c-bfa6-b0147d58a047 admin admin] PATH_INFO: `/v3/users/e7e15d7a479d40f0aa45226e765ecbaf` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:03.499796 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f234fec0-5c56-436c-bfa6-b0147d58a047 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7e15d7a479d40f0aa45226e765ecbaf)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:03.509834 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: WARNING py.warnings [None req-f234fec0-5c56-436c-bfa6-b0147d58a047 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 05:13:03.509834 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: warnings.warn(msg) Aug 03 05:13:03.509834 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: Aug 03 05:13:03.524305 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f234fec0-5c56-436c-bfa6-b0147d58a047 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:03.579295 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 856/1704] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:03 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 522 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:13:03.580948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93997d0-27b3-4079-9495-a47a3dac851c None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:03.584309 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93997d0-27b3-4079-9495-a47a3dac851c None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:03.586185 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93997d0-27b3-4079-9495-a47a3dac851c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:03.668037 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-c93997d0-27b3-4079-9495-a47a3dac851c None None] MFA Rules not processed for user `f01737ba3200470c8635f335d0338ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:03.669279 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-c93997d0-27b3-4079-9495-a47a3dac851c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:03.827474 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 857/1705] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:03.842227 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93997d0-27b3-4079-9495-a47a3dac851c None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:03.860018 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93997d0-27b3-4079-9495-a47a3dac851c None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:03.874565 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636ac36-a0fd-4061-a52f-a3bf8767d07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:03.875293 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636ac36-a0fd-4061-a52f-a3bf8767d07f admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:03.875576 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636ac36-a0fd-4061-a52f-a3bf8767d07f admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:03.875854 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636ac36-a0fd-4061-a52f-a3bf8767d07f admin admin] PATH_INFO: `/v3/projects/27b29ad85d3f431994f53598451a7a97/users/76f686ceb7ec41f1a4af40f06502575d/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:03.911221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636ac36-a0fd-4061-a52f-a3bf8767d07f admin admin] RBAC: Authorizing `identity:create_grant(project_id=27b29ad85d3f431994f53598451a7a97, user_id=76f686ceb7ec41f1a4af40f06502575d, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:03.916756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636ac36-a0fd-4061-a52f-a3bf8767d07f admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:03.964969 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f234fec0-5c56-436c-bfa6-b0147d58a047 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7e15d7a479d40f0aa45226e765ecbaf'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:04.090926 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 858/1706] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:03 2022] PUT /identity/v3/projects/27b29ad85d3f431994f53598451a7a97/users/76f686ceb7ec41f1a4af40f06502575d/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:04.099340 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636ac36-a0fd-4061-a52f-a3bf8767d07f admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:04.099340 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-c636ac36-a0fd-4061-a52f-a3bf8767d07f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:04.125118 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f234fec0-5c56-436c-bfa6-b0147d58a047 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7e15d7a479d40f0aa45226e765ecbaf'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:04.130875 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f234fec0-5c56-436c-bfa6-b0147d58a047 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7e15d7a479d40f0aa45226e765ecbaf'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:04.167222 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f234fec0-5c56-436c-bfa6-b0147d58a047 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7e15d7a479d40f0aa45226e765ecbaf'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:04.168053 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f234fec0-5c56-436c-bfa6-b0147d58a047 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7e15d7a479d40f0aa45226e765ecbaf'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:04.183208 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636ac36-a0fd-4061-a52f-a3bf8767d07f admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:04.185085 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 849/1707] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:13:03 2022] DELETE /identity/v3/users/e7e15d7a479d40f0aa45226e765ecbaf => generated 0 bytes in 804 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:04.185729 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f234fec0-5c56-436c-bfa6-b0147d58a047 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:04.187199 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f234fec0-5c56-436c-bfa6-b0147d58a047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:04.266596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:04.278960 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5fba0-8110-449d-91dd-0cc69607033f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:04.283027 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5fba0-8110-449d-91dd-0cc69607033f service cinder] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:04.283027 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5fba0-8110-449d-91dd-0cc69607033f service cinder] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:04.283027 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5fba0-8110-449d-91dd-0cc69607033f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:04.283731 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-1df5fba0-8110-449d-91dd-0cc69607033f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:04.332294 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f234fec0-5c56-436c-bfa6-b0147d58a047 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:04.352911 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480aaf2a-3983-4ec6-b8f6-13199c7915a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:04.353994 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480aaf2a-3983-4ec6-b8f6-13199c7915a0 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:04.353994 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480aaf2a-3983-4ec6-b8f6-13199c7915a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:04.354161 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480aaf2a-3983-4ec6-b8f6-13199c7915a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:04.365125 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df5fba0-8110-449d-91dd-0cc69607033f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:04.366356 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-480aaf2a-3983-4ec6-b8f6-13199c7915a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:04.367436 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df5fba0-8110-449d-91dd-0cc69607033f service cinder] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:04.379408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 859/1708] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:04 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:04.381202 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5fba0-8110-449d-91dd-0cc69607033f service cinder] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:04.383247 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-1df5fba0-8110-449d-91dd-0cc69607033f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:04.583455 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480aaf2a-3983-4ec6-b8f6-13199c7915a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:04.588873 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480aaf2a-3983-4ec6-b8f6-13199c7915a0 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:04.613121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 850/1709] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:04.615258 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a49d6-6b58-42a1-a4d5-2fc1271602b6 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:04.616476 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a49d6-6b58-42a1-a4d5-2fc1271602b6 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:04.616905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a49d6-6b58-42a1-a4d5-2fc1271602b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:04.625139 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5fba0-8110-449d-91dd-0cc69607033f service cinder] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:04.662263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a95808-94d7-43e9-acd2-1dc45a21b217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:04.663203 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a95808-94d7-43e9-acd2-1dc45a21b217 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:04.665731 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a95808-94d7-43e9-acd2-1dc45a21b217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:04.670938 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a95808-94d7-43e9-acd2-1dc45a21b217 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:04.671905 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a95808-94d7-43e9-acd2-1dc45a21b217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:04.677734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a95808-94d7-43e9-acd2-1dc45a21b217 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:04.730154 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-7f0a49d6-6b58-42a1-a4d5-2fc1271602b6 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:04.738679 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7f0a49d6-6b58-42a1-a4d5-2fc1271602b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:04.760226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 860/1710] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:04.766293 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 851/1711] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:04.767456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a95808-94d7-43e9-acd2-1dc45a21b217 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:04.793134 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a95808-94d7-43e9-acd2-1dc45a21b217 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:04.804818 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7ee0d-f380-4ed0-8673-10dc7ff32b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:04.805865 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7ee0d-f380-4ed0-8673-10dc7ff32b31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:04.806349 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7ee0d-f380-4ed0-8673-10dc7ff32b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:04.806821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7ee0d-f380-4ed0-8673-10dc7ff32b31 admin admin] PATH_INFO: `/v3/projects/27b29ad85d3f431994f53598451a7a97/users/76f686ceb7ec41f1a4af40f06502575d/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:04.811767 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7ee0d-f380-4ed0-8673-10dc7ff32b31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27b29ad85d3f431994f53598451a7a97, user_id=76f686ceb7ec41f1a4af40f06502575d, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:04.815152 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7ee0d-f380-4ed0-8673-10dc7ff32b31 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:04.857702 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 861/1712] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:04 2022] PUT /identity/v3/projects/27b29ad85d3f431994f53598451a7a97/users/76f686ceb7ec41f1a4af40f06502575d/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:04.863907 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebced198-5450-481e-ab1b-2cee7f839387 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:04.865043 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebced198-5450-481e-ab1b-2cee7f839387 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:04.865897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebced198-5450-481e-ab1b-2cee7f839387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:04.912405 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7ee0d-f380-4ed0-8673-10dc7ff32b31 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:04.914378 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-02b7ee0d-f380-4ed0-8673-10dc7ff32b31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:05.003734 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-ebced198-5450-481e-ab1b-2cee7f839387 None None] MFA Rules not processed for user `76f686ceb7ec41f1a4af40f06502575d`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:05.005704 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ebced198-5450-481e-ab1b-2cee7f839387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:05.119536 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7ee0d-f380-4ed0-8673-10dc7ff32b31 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:05.127905 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a65bc7-3c17-4cc9-846e-2ed681618652 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:05.130270 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a65bc7-3c17-4cc9-846e-2ed681618652 service cinder] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:05.131433 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a65bc7-3c17-4cc9-846e-2ed681618652 service cinder] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:05.132588 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a65bc7-3c17-4cc9-846e-2ed681618652 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:05.147961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-60a65bc7-3c17-4cc9-846e-2ed681618652 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:05.156839 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:05.234330 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 852/1713] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:04 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:05.312180 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a65bc7-3c17-4cc9-846e-2ed681618652 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:05.315179 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a65bc7-3c17-4cc9-846e-2ed681618652 service cinder] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:05.329314 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 862/1714] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:13:05.356992 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebced198-5450-481e-ab1b-2cee7f839387 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:05.358441 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ebced198-5450-481e-ab1b-2cee7f839387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:05.418322 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a65bc7-3c17-4cc9-846e-2ed681618652 service cinder] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:05.421212 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-60a65bc7-3c17-4cc9-846e-2ed681618652 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:05.435771 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a65bc7-3c17-4cc9-846e-2ed681618652 service cinder] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:05.444487 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e279656-2adf-4fd1-a62f-32f7d984f4b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:05.449779 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e279656-2adf-4fd1-a62f-32f7d984f4b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:05.449779 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e279656-2adf-4fd1-a62f-32f7d984f4b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:05.449779 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e279656-2adf-4fd1-a62f-32f7d984f4b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:05.464131 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e279656-2adf-4fd1-a62f-32f7d984f4b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:05.465877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e279656-2adf-4fd1-a62f-32f7d984f4b7 service cinder] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:05.511021 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 863/1715] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:05 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:13:05.565254 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:05.578589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebced198-5450-481e-ab1b-2cee7f839387 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:05.591217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99971832-4fce-4191-89d1-4f24ae635b93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:05.591994 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99971832-4fce-4191-89d1-4f24ae635b93 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:05.592289 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99971832-4fce-4191-89d1-4f24ae635b93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:05.592575 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99971832-4fce-4191-89d1-4f24ae635b93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:05.618133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de906b-668d-4ba8-8e19-e20b7096544c None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:05.619205 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de906b-668d-4ba8-8e19-e20b7096544c None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:05.619378 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de906b-668d-4ba8-8e19-e20b7096544c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:05.722701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-49de906b-668d-4ba8-8e19-e20b7096544c None None] MFA Rules not processed for user `41541854b74c49a886db2e7bfa867648`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:05.724127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-49de906b-668d-4ba8-8e19-e20b7096544c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:05.850519 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99971832-4fce-4191-89d1-4f24ae635b93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:05.853895 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99971832-4fce-4191-89d1-4f24ae635b93 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:05.887594 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 853/1716] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:05.891523 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af301738-8384-41e4-aa76-1c1b1f95deb3 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:05.892118 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af301738-8384-41e4-aa76-1c1b1f95deb3 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:05.893183 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af301738-8384-41e4-aa76-1c1b1f95deb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:05.951604 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 864/1717] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:05.954661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2da279-70a7-420f-b6e9-2554ade8c4ba None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:05.954987 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2da279-70a7-420f-b6e9-2554ade8c4ba None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:05.955269 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2da279-70a7-420f-b6e9-2554ade8c4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:06.030362 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-0b2da279-70a7-420f-b6e9-2554ade8c4ba None None] MFA Rules not processed for user `41541854b74c49a886db2e7bfa867648`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:06.047322 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-0b2da279-70a7-420f-b6e9-2554ade8c4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:06.077962 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 865/1718] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:06.093694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2da279-70a7-420f-b6e9-2554ade8c4ba None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:06.141128 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-af301738-8384-41e4-aa76-1c1b1f95deb3 None None] MFA Rules not processed for user `de853a2dfffb4f05b6f47df2280ae4e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:06.144172 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-af301738-8384-41e4-aa76-1c1b1f95deb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:06.408714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 854/1719] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Aug 3 05:13:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 520 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:06.434911 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:06.440861 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2da279-70a7-420f-b6e9-2554ade8c4ba None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:06.456908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af301738-8384-41e4-aa76-1c1b1f95deb3 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:06.460011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-af301738-8384-41e4-aa76-1c1b1f95deb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:06.465906 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1133e6c-fe3b-47ea-966e-c544546492a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:06.467732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1133e6c-fe3b-47ea-966e-c544546492a5 service nova] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:06.470966 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1133e6c-fe3b-47ea-966e-c544546492a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:06.471596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1133e6c-fe3b-47ea-966e-c544546492a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:06.510123 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1133e6c-fe3b-47ea-966e-c544546492a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:06.513697 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1133e6c-fe3b-47ea-966e-c544546492a5 service nova] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:06.538214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 866/1720] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:06.576175 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af301738-8384-41e4-aa76-1c1b1f95deb3 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:06.634649 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84c62f-5ba2-41b7-8b0a-00fb341c20c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:06.650876 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84c62f-5ba2-41b7-8b0a-00fb341c20c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:06.650876 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84c62f-5ba2-41b7-8b0a-00fb341c20c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:06.650876 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84c62f-5ba2-41b7-8b0a-00fb341c20c3 admin admin] PATH_INFO: `/v3/users/77af257997f44fb5b5223c5d9af87b00` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:06.650876 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d84c62f-5ba2-41b7-8b0a-00fb341c20c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77af257997f44fb5b5223c5d9af87b00)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:06.667288 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d84c62f-5ba2-41b7-8b0a-00fb341c20c3 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:06.737297 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fc853-8970-4b2c-9e95-b0177a900e29 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:06.742813 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fc853-8970-4b2c-9e95-b0177a900e29 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:06.749348 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fc853-8970-4b2c-9e95-b0177a900e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:06.804579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-be0fc853-8970-4b2c-9e95-b0177a900e29 None None] MFA Rules not processed for user `747a608061504d7693f0f99ecdfa476e`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:06.807519 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-be0fc853-8970-4b2c-9e95-b0177a900e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:06.912745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 867/1721] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:06 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:13:06.960913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0fc853-8970-4b2c-9e95-b0177a900e29 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:06.963005 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-be0fc853-8970-4b2c-9e95-b0177a900e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:06.996271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8d84c62f-5ba2-41b7-8b0a-00fb341c20c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77af257997f44fb5b5223c5d9af87b00'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:07.107766 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8d84c62f-5ba2-41b7-8b0a-00fb341c20c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77af257997f44fb5b5223c5d9af87b00'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:07.108449 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8d84c62f-5ba2-41b7-8b0a-00fb341c20c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77af257997f44fb5b5223c5d9af87b00'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:07.113019 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:07.139392 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8d84c62f-5ba2-41b7-8b0a-00fb341c20c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77af257997f44fb5b5223c5d9af87b00'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:07.140266 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8d84c62f-5ba2-41b7-8b0a-00fb341c20c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77af257997f44fb5b5223c5d9af87b00'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:07.165722 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 855/1722] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:13:06 2022] DELETE /identity/v3/users/77af257997f44fb5b5223c5d9af87b00 => generated 0 bytes in 712 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:07.170354 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea73e5-3d62-4288-96be-993f3cdb9e2e None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:07.170796 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea73e5-3d62-4288-96be-993f3cdb9e2e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:07.171314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea73e5-3d62-4288-96be-993f3cdb9e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:07.194769 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-4aea73e5-3d62-4288-96be-993f3cdb9e2e None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:07.196821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-4aea73e5-3d62-4288-96be-993f3cdb9e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:07.220263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0fc853-8970-4b2c-9e95-b0177a900e29 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:07.326911 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 856/1723] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:07.366422 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea73e5-3d62-4288-96be-993f3cdb9e2e None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:07.367943 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-4aea73e5-3d62-4288-96be-993f3cdb9e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:07.386229 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd1129-180b-472c-81e4-f1e751b23477 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1014bcb4d36441aa46042941c9a63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:07.386896 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd1129-180b-472c-81e4-f1e751b23477 service swift] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:07.387194 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd1129-180b-472c-81e4-f1e751b23477 service swift] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:07.387479 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd1129-180b-472c-81e4-f1e751b23477 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:07.390932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-e8dd1129-180b-472c-81e4-f1e751b23477 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:07.548903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dd1129-180b-472c-81e4-f1e751b23477 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:07.550558 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dd1129-180b-472c-81e4-f1e751b23477 service swift] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:07.554262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea73e5-3d62-4288-96be-993f3cdb9e2e None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:07.559386 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657b394-b720-49bd-add0-c5fcd6f3e12b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:07.560335 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657b394-b720-49bd-add0-c5fcd6f3e12b service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:07.560763 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657b394-b720-49bd-add0-c5fcd6f3e12b service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:07.561362 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657b394-b720-49bd-add0-c5fcd6f3e12b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:07.585306 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657b394-b720-49bd-add0-c5fcd6f3e12b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:07.587115 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657b394-b720-49bd-add0-c5fcd6f3e12b service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:07.589884 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 868/1724] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 3 05:13:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 05:13:07.639293 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 857/1725] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:07.778238 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae19f69-0f59-4158-b152-90b2db041108 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:07.781217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae19f69-0f59-4158-b152-90b2db041108 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:07.781664 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae19f69-0f59-4158-b152-90b2db041108 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:07.784093 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 869/1726] 10.222.0.40 () {54 vars in 912 bytes} [Wed Aug 3 05:13:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:13:07.793174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeee315-8b11-4e04-9067-1ed132b1f287 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:07.794004 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeee315-8b11-4e04-9067-1ed132b1f287 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:07.794469 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeee315-8b11-4e04-9067-1ed132b1f287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:07.881780 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-feeee315-8b11-4e04-9067-1ed132b1f287 None None] MFA Rules not processed for user `cff15ea4cbc64b068aa5f6586b5bff87`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:07.885457 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-feeee315-8b11-4e04-9067-1ed132b1f287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:07.933987 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 858/1727] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Aug 3 05:13:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:08.062059 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae19f69-0f59-4158-b152-90b2db041108 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:08.065961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-7ae19f69-0f59-4158-b152-90b2db041108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:08.088164 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeee315-8b11-4e04-9067-1ed132b1f287 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:08.099829 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeee315-8b11-4e04-9067-1ed132b1f287 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:08.115673 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72ce1a-9688-4170-9801-7489c2c900e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:08.122584 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72ce1a-9688-4170-9801-7489c2c900e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:08.129999 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72ce1a-9688-4170-9801-7489c2c900e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:08.129999 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72ce1a-9688-4170-9801-7489c2c900e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:08.139740 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae19f69-0f59-4158-b152-90b2db041108 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:08.195806 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a72ce1a-9688-4170-9801-7489c2c900e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:08.196268 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf76b7-948d-4776-91f2-a499be5779b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:08.204241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf76b7-948d-4776-91f2-a499be5779b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:08.204717 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf76b7-948d-4776-91f2-a499be5779b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:08.205016 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf76b7-948d-4776-91f2-a499be5779b8 admin admin] PATH_INFO: `/v3/users/ae5fa3f9e36940d3be189260adfe3321` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:08.209430 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a72ce1a-9688-4170-9801-7489c2c900e3 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:08.213253 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf76b7-948d-4776-91f2-a499be5779b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae5fa3f9e36940d3be189260adfe3321)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:08.215410 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf76b7-948d-4776-91f2-a499be5779b8 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:08.252146 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 859/1728] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:13:08.558189 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:08.585321 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-7aaf76b7-948d-4776-91f2-a499be5779b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae5fa3f9e36940d3be189260adfe3321'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:08.618539 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:08.672758 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-7aaf76b7-948d-4776-91f2-a499be5779b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae5fa3f9e36940d3be189260adfe3321'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:08.673572 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-7aaf76b7-948d-4776-91f2-a499be5779b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae5fa3f9e36940d3be189260adfe3321'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:08.705729 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-7aaf76b7-948d-4776-91f2-a499be5779b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae5fa3f9e36940d3be189260adfe3321'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:08.707164 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-7aaf76b7-948d-4776-91f2-a499be5779b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae5fa3f9e36940d3be189260adfe3321'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:08.759607 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 870/1729] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:13:08 2022] DELETE /identity/v3/users/ae5fa3f9e36940d3be189260adfe3321 => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:08.764973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9675e-7b87-40a1-bb67-6dd649d50fc7 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:08.765445 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9675e-7b87-40a1-bb67-6dd649d50fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:08.766079 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9675e-7b87-40a1-bb67-6dd649d50fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:08.821678 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-bba9675e-7b87-40a1-bb67-6dd649d50fc7 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:08.831022 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-bba9675e-7b87-40a1-bb67-6dd649d50fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:09.022509 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 860/1730] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:09.142149 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf76b7-948d-4776-91f2-a499be5779b8 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:09.142149 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-7aaf76b7-948d-4776-91f2-a499be5779b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:09.292592 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf76b7-948d-4776-91f2-a499be5779b8 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:09.299771 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858cee79-f156-48a7-8555-80de6fb4cc05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:09.299771 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858cee79-f156-48a7-8555-80de6fb4cc05 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:09.299771 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858cee79-f156-48a7-8555-80de6fb4cc05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:09.299771 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858cee79-f156-48a7-8555-80de6fb4cc05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:09.330391 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858cee79-f156-48a7-8555-80de6fb4cc05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:09.339525 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858cee79-f156-48a7-8555-80de6fb4cc05 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:09.354217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 871/1731] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:09.575764 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba9675e-7b87-40a1-bb67-6dd649d50fc7 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:09.577213 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-bba9675e-7b87-40a1-bb67-6dd649d50fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:09.603980 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba9675e-7b87-40a1-bb67-6dd649d50fc7 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:09.613680 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918f58e-ebe0-4fa5-916d-ab5ba82195b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:09.615033 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918f58e-ebe0-4fa5-916d-ab5ba82195b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:09.615497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918f58e-ebe0-4fa5-916d-ab5ba82195b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:09.615936 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918f58e-ebe0-4fa5-916d-ab5ba82195b0 admin admin] PATH_INFO: `/v3/projects/42d98a9a72b142729e08ab431290358f` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:09.617406 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d918f58e-ebe0-4fa5-916d-ab5ba82195b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42d98a9a72b142729e08ab431290358f)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:09.619565 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d918f58e-ebe0-4fa5-916d-ab5ba82195b0 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:09.838489 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d918f58e-ebe0-4fa5-916d-ab5ba82195b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42d98a9a72b142729e08ab431290358f', 'request_id': 'req-d918f58e-ebe0-4fa5-916d-ab5ba82195b0'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:09.872947 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:09.919709 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d918f58e-ebe0-4fa5-916d-ab5ba82195b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42d98a9a72b142729e08ab431290358f', 'request_id': 'req-d918f58e-ebe0-4fa5-916d-ab5ba82195b0'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:09.927711 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d918f58e-ebe0-4fa5-916d-ab5ba82195b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42d98a9a72b142729e08ab431290358f', 'request_id': 'req-d918f58e-ebe0-4fa5-916d-ab5ba82195b0'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:09.956661 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:09.995749 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d918f58e-ebe0-4fa5-916d-ab5ba82195b0 admin admin] The token cache is being invalidate because project 42d98a9a72b142729e08ab431290358f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:13:09.996318 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d918f58e-ebe0-4fa5-916d-ab5ba82195b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:10.005419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d918f58e-ebe0-4fa5-916d-ab5ba82195b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:10.011378 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6627b-8079-43de-8ddc-3ade6afeea79 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:10.011892 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6627b-8079-43de-8ddc-3ade6afeea79 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:10.019686 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6627b-8079-43de-8ddc-3ade6afeea79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:10.019813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 861/1732] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:13:09 2022] DELETE /identity/v3/projects/42d98a9a72b142729e08ab431290358f => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:10.019813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918f58e-ebe0-4fa5-916d-ab5ba82195b0 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:10.027826 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d918f58e-ebe0-4fa5-916d-ab5ba82195b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:10.105768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-50e6627b-8079-43de-8ddc-3ade6afeea79 None None] MFA Rules not processed for user `cff15ea4cbc64b068aa5f6586b5bff87`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:10.134825 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-50e6627b-8079-43de-8ddc-3ade6afeea79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:10.399198 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918f58e-ebe0-4fa5-916d-ab5ba82195b0 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:10.424529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f147a7-4427-49f4-a381-60e7512dd104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:10.430914 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f147a7-4427-49f4-a381-60e7512dd104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:10.431504 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f147a7-4427-49f4-a381-60e7512dd104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:10.432394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f147a7-4427-49f4-a381-60e7512dd104 admin admin] PATH_INFO: `/v3/users/cf1b9c109e924ce8899647ebe7a589da` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:10.435804 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f147a7-4427-49f4-a381-60e7512dd104 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf1b9c109e924ce8899647ebe7a589da)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:10.440743 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f147a7-4427-49f4-a381-60e7512dd104 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:10.455081 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 872/1733] 10.222.0.67 () {58 vars in 1015 bytes} [Wed Aug 3 05:13:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 445 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:10.519871 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:10.600651 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:10.709319 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6627b-8079-43de-8ddc-3ade6afeea79 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:10.712783 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-50e6627b-8079-43de-8ddc-3ade6afeea79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:10.744612 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e7f147a7-4427-49f4-a381-60e7512dd104 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf1b9c109e924ce8899647ebe7a589da'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:10.746096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6627b-8079-43de-8ddc-3ade6afeea79 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:10.767575 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d86b7-c9e2-4ba2-ae57-e260fccc6d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:10.772696 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d86b7-c9e2-4ba2-ae57-e260fccc6d6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:10.776158 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d86b7-c9e2-4ba2-ae57-e260fccc6d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:10.776723 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d86b7-c9e2-4ba2-ae57-e260fccc6d6b admin admin] PATH_INFO: `/v3/users/b967b5aa62254fc0b086344bccb041b8` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:10.795377 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9d86b7-c9e2-4ba2-ae57-e260fccc6d6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b967b5aa62254fc0b086344bccb041b8)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:10.797351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9d86b7-c9e2-4ba2-ae57-e260fccc6d6b admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:10.875646 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e7f147a7-4427-49f4-a381-60e7512dd104 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf1b9c109e924ce8899647ebe7a589da'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:10.882495 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e7f147a7-4427-49f4-a381-60e7512dd104 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf1b9c109e924ce8899647ebe7a589da'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:10.944454 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e7f147a7-4427-49f4-a381-60e7512dd104 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf1b9c109e924ce8899647ebe7a589da'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:10.946292 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e7f147a7-4427-49f4-a381-60e7512dd104 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf1b9c109e924ce8899647ebe7a589da'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:10.989534 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 862/1734] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:13:10 2022] DELETE /identity/v3/users/cf1b9c109e924ce8899647ebe7a589da => generated 0 bytes in 969 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:10.995249 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f147a7-4427-49f4-a381-60e7512dd104 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:11.003211 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e7f147a7-4427-49f4-a381-60e7512dd104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:11.193830 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f147a7-4427-49f4-a381-60e7512dd104 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:11.213802 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-dc9d86b7-c9e2-4ba2-ae57-e260fccc6d6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b967b5aa62254fc0b086344bccb041b8'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:11.223867 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4b944-9c34-4f02-96e8-316ecbbe2b3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:11.224813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4b944-9c34-4f02-96e8-316ecbbe2b3b service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:11.225097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4b944-9c34-4f02-96e8-316ecbbe2b3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:11.225372 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4b944-9c34-4f02-96e8-316ecbbe2b3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:11.231380 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3ce4b944-9c34-4f02-96e8-316ecbbe2b3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:11.302317 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce4b944-9c34-4f02-96e8-316ecbbe2b3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:11.302317 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce4b944-9c34-4f02-96e8-316ecbbe2b3b service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:11.319847 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-dc9d86b7-c9e2-4ba2-ae57-e260fccc6d6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b967b5aa62254fc0b086344bccb041b8'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:11.320766 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-dc9d86b7-c9e2-4ba2-ae57-e260fccc6d6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b967b5aa62254fc0b086344bccb041b8'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:11.352470 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-dc9d86b7-c9e2-4ba2-ae57-e260fccc6d6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b967b5aa62254fc0b086344bccb041b8'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:11.355976 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-dc9d86b7-c9e2-4ba2-ae57-e260fccc6d6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b967b5aa62254fc0b086344bccb041b8'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:11.370032 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 863/1735] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:11.373317 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4b944-9c34-4f02-96e8-316ecbbe2b3b service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:11.376946 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3ce4b944-9c34-4f02-96e8-316ecbbe2b3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:11.421939 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 873/1736] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:13:10 2022] DELETE /identity/v3/users/b967b5aa62254fc0b086344bccb041b8 => generated 0 bytes in 713 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:11.428536 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36605f1-2670-453a-9807-efed626837ff None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:11.429443 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36605f1-2670-453a-9807-efed626837ff None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:11.432777 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36605f1-2670-453a-9807-efed626837ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:11.552934 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-a36605f1-2670-453a-9807-efed626837ff None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:11.555085 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-a36605f1-2670-453a-9807-efed626837ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:11.598609 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4b944-9c34-4f02-96e8-316ecbbe2b3b service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:11.794222 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 874/1737] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:11.794222 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36605f1-2670-453a-9807-efed626837ff None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:11.794222 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-a36605f1-2670-453a-9807-efed626837ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:11.868289 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0081e4-c261-4f2f-b0bc-e0ad42958284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:11.869929 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0081e4-c261-4f2f-b0bc-e0ad42958284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:11.870475 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0081e4-c261-4f2f-b0bc-e0ad42958284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:11.870984 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0081e4-c261-4f2f-b0bc-e0ad42958284 admin admin] PATH_INFO: `/v3/projects/9cd419ba699d4449a25fd90a89de214d` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:11.877282 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0081e4-c261-4f2f-b0bc-e0ad42958284 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cd419ba699d4449a25fd90a89de214d)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:11.880512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0081e4-c261-4f2f-b0bc-e0ad42958284 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:11.980947 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36605f1-2670-453a-9807-efed626837ff None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:11.998815 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df7404-1aeb-4dfa-9ef4-8b79783586e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:11.999777 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df7404-1aeb-4dfa-9ef4-8b79783586e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:12.000326 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df7404-1aeb-4dfa-9ef4-8b79783586e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:12.000947 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df7404-1aeb-4dfa-9ef4-8b79783586e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:12.003476 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-f1df7404-1aeb-4dfa-9ef4-8b79783586e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:12.026572 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df7404-1aeb-4dfa-9ef4-8b79783586e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:12.043842 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df7404-1aeb-4dfa-9ef4-8b79783586e5 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:12.066441 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 875/1738] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:12.068146 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df7404-1aeb-4dfa-9ef4-8b79783586e5 service neutron] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:12.073718 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-f1df7404-1aeb-4dfa-9ef4-8b79783586e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:12.095850 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df7404-1aeb-4dfa-9ef4-8b79783586e5 service neutron] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:12.098420 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-6c0081e4-c261-4f2f-b0bc-e0ad42958284 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cd419ba699d4449a25fd90a89de214d', 'request_id': 'req-6c0081e4-c261-4f2f-b0bc-e0ad42958284'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:12.171433 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-6c0081e4-c261-4f2f-b0bc-e0ad42958284 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cd419ba699d4449a25fd90a89de214d', 'request_id': 'req-6c0081e4-c261-4f2f-b0bc-e0ad42958284'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:12.180859 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-6c0081e4-c261-4f2f-b0bc-e0ad42958284 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cd419ba699d4449a25fd90a89de214d', 'request_id': 'req-6c0081e4-c261-4f2f-b0bc-e0ad42958284'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:12.221736 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-6c0081e4-c261-4f2f-b0bc-e0ad42958284 admin admin] The token cache is being invalidate because project 9cd419ba699d4449a25fd90a89de214d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:13:12.222259 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-6c0081e4-c261-4f2f-b0bc-e0ad42958284 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:12.223615 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-6c0081e4-c261-4f2f-b0bc-e0ad42958284 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:12.228718 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 864/1739] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:13:11 2022] DELETE /identity/v3/projects/9cd419ba699d4449a25fd90a89de214d => generated 0 bytes in 857 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:12.233769 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa73be9-cc02-4df5-bebd-17316008ecae None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:12.234285 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa73be9-cc02-4df5-bebd-17316008ecae None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:12.234776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa73be9-cc02-4df5-bebd-17316008ecae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:12.267533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ac589-382a-4660-82b4-422fe8c87008 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:12.269483 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ac589-382a-4660-82b4-422fe8c87008 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:12.272333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ac589-382a-4660-82b4-422fe8c87008 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:12.272810 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ac589-382a-4660-82b4-422fe8c87008 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:12.289843 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-e78ac589-382a-4660-82b4-422fe8c87008 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:12.327429 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-afa73be9-cc02-4df5-bebd-17316008ecae None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:12.334919 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-afa73be9-cc02-4df5-bebd-17316008ecae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:12.355498 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:12.481813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 865/1740] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:12.488944 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ac589-382a-4660-82b4-422fe8c87008 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:12.488944 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ac589-382a-4660-82b4-422fe8c87008 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:12.490339 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2aacf2-0a52-454e-9858-96a57b85198a None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:12.490988 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2aacf2-0a52-454e-9858-96a57b85198a None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:12.491342 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2aacf2-0a52-454e-9858-96a57b85198a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:12.502337 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 876/1741] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:12.523342 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:12.539515 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-8a2aacf2-0a52-454e-9858-96a57b85198a None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:12.541069 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8a2aacf2-0a52-454e-9858-96a57b85198a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:12.552225 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 866/1742] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:13:12.556244 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ac589-382a-4660-82b4-422fe8c87008 service neutron] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:12.570057 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ac589-382a-4660-82b4-422fe8c87008 service neutron] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:12.578351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab4e15-a53c-4380-8888-6556b10f7e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:12.579850 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab4e15-a53c-4380-8888-6556b10f7e16 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:12.580346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab4e15-a53c-4380-8888-6556b10f7e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:12.580850 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab4e15-a53c-4380-8888-6556b10f7e16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:12.583211 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ab4e15-a53c-4380-8888-6556b10f7e16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:13:12.583759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ab4e15-a53c-4380-8888-6556b10f7e16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:12.586118 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ab4e15-a53c-4380-8888-6556b10f7e16 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:12.609638 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 877/1743] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 05:13:12.619002 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2aacf2-0a52-454e-9858-96a57b85198a None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:12.653318 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2aacf2-0a52-454e-9858-96a57b85198a None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:12.671434 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc57ef-c05d-455e-a6a0-bf9e26344fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:12.673301 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc57ef-c05d-455e-a6a0-bf9e26344fd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:12.673698 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc57ef-c05d-455e-a6a0-bf9e26344fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:12.674144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc57ef-c05d-455e-a6a0-bf9e26344fd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:12.675536 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cc57ef-c05d-455e-a6a0-bf9e26344fd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:12.679500 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cc57ef-c05d-455e-a6a0-bf9e26344fd5 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:12.812726 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 867/1744] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:13:12 2022] POST /identity/v3/projects => generated 365 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:12.819990 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab4e15-a53c-4380-8888-6556b10f7e16 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:12.855091 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab4e15-a53c-4380-8888-6556b10f7e16 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:13.013735 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e42be-cbc7-4107-9a63-fc33978ae3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:13.018424 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e42be-cbc7-4107-9a63-fc33978ae3f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:13.018424 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e42be-cbc7-4107-9a63-fc33978ae3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:13.018424 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e42be-cbc7-4107-9a63-fc33978ae3f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:13.023594 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e42be-cbc7-4107-9a63-fc33978ae3f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:13.025761 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e42be-cbc7-4107-9a63-fc33978ae3f0 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:13.124476 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc57ef-c05d-455e-a6a0-bf9e26344fd5 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:13.130334 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a8cc57ef-c05d-455e-a6a0-bf9e26344fd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:13.171250 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc57ef-c05d-455e-a6a0-bf9e26344fd5 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:13.177707 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 878/1745] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:13:12 2022] POST /identity/v3/users => generated 283 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:13.183169 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e42be-cbc7-4107-9a63-fc33978ae3f0 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:13.183305 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db72ee7-63dd-41f5-bdaf-3d73fd6a6773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:13.191342 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db72ee7-63dd-41f5-bdaf-3d73fd6a6773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:13.194285 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db72ee7-63dd-41f5-bdaf-3d73fd6a6773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:13.198011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db72ee7-63dd-41f5-bdaf-3d73fd6a6773 admin admin] PATH_INFO: `/v3/projects/15d57423afdc4b6c8b3a2e9d3e6d5328` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:13.199723 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db72ee7-63dd-41f5-bdaf-3d73fd6a6773 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15d57423afdc4b6c8b3a2e9d3e6d5328)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:13.202033 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db72ee7-63dd-41f5-bdaf-3d73fd6a6773 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:13.204673 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e42be-cbc7-4107-9a63-fc33978ae3f0 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:13.230324 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57fa7e-08b1-43f2-be37-6f08bbaf72d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:13.231055 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57fa7e-08b1-43f2-be37-6f08bbaf72d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:13.231346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57fa7e-08b1-43f2-be37-6f08bbaf72d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:13.231630 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57fa7e-08b1-43f2-be37-6f08bbaf72d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:13.232521 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed57fa7e-08b1-43f2-be37-6f08bbaf72d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:13.235276 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed57fa7e-08b1-43f2-be37-6f08bbaf72d9 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:13.273368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 879/1746] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:13:13.289389 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57fa7e-08b1-43f2-be37-6f08bbaf72d9 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:13.334230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57fa7e-08b1-43f2-be37-6f08bbaf72d9 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:13.341439 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32709ceb-290d-403d-bfe9-18b876f3d36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:13.342300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32709ceb-290d-403d-bfe9-18b876f3d36d admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:13.342749 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32709ceb-290d-403d-bfe9-18b876f3d36d admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:13.343235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32709ceb-290d-403d-bfe9-18b876f3d36d admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:13.344750 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32709ceb-290d-403d-bfe9-18b876f3d36d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:13.351593 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32709ceb-290d-403d-bfe9-18b876f3d36d admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:13.399213 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 880/1747] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:13.404373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32709ceb-290d-403d-bfe9-18b876f3d36d admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:13.420696 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-32709ceb-290d-403d-bfe9-18b876f3d36d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:13.475040 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32709ceb-290d-403d-bfe9-18b876f3d36d admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:13.485094 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7db72ee7-63dd-41f5-bdaf-3d73fd6a6773 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15d57423afdc4b6c8b3a2e9d3e6d5328', 'request_id': 'req-7db72ee7-63dd-41f5-bdaf-3d73fd6a6773'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:13.595294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a9888-b1e8-4317-8311-c2d85a75421a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:13.598182 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7db72ee7-63dd-41f5-bdaf-3d73fd6a6773 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15d57423afdc4b6c8b3a2e9d3e6d5328', 'request_id': 'req-7db72ee7-63dd-41f5-bdaf-3d73fd6a6773'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:13.599906 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a9888-b1e8-4317-8311-c2d85a75421a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:13.600271 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a9888-b1e8-4317-8311-c2d85a75421a admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:13.600608 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a9888-b1e8-4317-8311-c2d85a75421a admin admin] PATH_INFO: `/v3/projects/5eaf5c3022d449748376965ad6ab87f2` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:13.603045 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a9888-b1e8-4317-8311-c2d85a75421a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eaf5c3022d449748376965ad6ab87f2)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:13.604070 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: WARNING py.warnings [None req-455a9888-b1e8-4317-8311-c2d85a75421a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 05:13:13.604070 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: warnings.warn(msg) Aug 03 05:13:13.604070 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: Aug 03 05:13:13.605606 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a9888-b1e8-4317-8311-c2d85a75421a admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:13.606589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7db72ee7-63dd-41f5-bdaf-3d73fd6a6773 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15d57423afdc4b6c8b3a2e9d3e6d5328', 'request_id': 'req-7db72ee7-63dd-41f5-bdaf-3d73fd6a6773'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:13.645497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7db72ee7-63dd-41f5-bdaf-3d73fd6a6773 admin admin] The token cache is being invalidate because project 15d57423afdc4b6c8b3a2e9d3e6d5328 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:13:13.649936 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7db72ee7-63dd-41f5-bdaf-3d73fd6a6773 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:13.649936 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7db72ee7-63dd-41f5-bdaf-3d73fd6a6773 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:13.649936 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 868/1748] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:13:13 2022] DELETE /identity/v3/projects/15d57423afdc4b6c8b3a2e9d3e6d5328 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:13.649936 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db72ee7-63dd-41f5-bdaf-3d73fd6a6773 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:13.652575 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7db72ee7-63dd-41f5-bdaf-3d73fd6a6773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:13.689132 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db72ee7-63dd-41f5-bdaf-3d73fd6a6773 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:13.689132 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fcbbb1-5c06-4f14-bfd5-6f1bdd479ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:13.690213 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fcbbb1-5c06-4f14-bfd5-6f1bdd479ec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:13.690547 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fcbbb1-5c06-4f14-bfd5-6f1bdd479ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:13.691016 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fcbbb1-5c06-4f14-bfd5-6f1bdd479ec2 admin admin] PATH_INFO: `/v3/projects/7f816875cdf441c08e8578f95c51f8b5/users/7b4c766f29e045a284f775220b58cda3/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:13.715598 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fcbbb1-5c06-4f14-bfd5-6f1bdd479ec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f816875cdf441c08e8578f95c51f8b5, user_id=7b4c766f29e045a284f775220b58cda3, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:13.718521 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fcbbb1-5c06-4f14-bfd5-6f1bdd479ec2 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:13.842934 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 869/1749] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:13 2022] PUT /identity/v3/projects/7f816875cdf441c08e8578f95c51f8b5/users/7b4c766f29e045a284f775220b58cda3/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:13.842934 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fcbbb1-5c06-4f14-bfd5-6f1bdd479ec2 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:13.852868 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b3fcbbb1-5c06-4f14-bfd5-6f1bdd479ec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:13.923905 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-455a9888-b1e8-4317-8311-c2d85a75421a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eaf5c3022d449748376965ad6ab87f2', 'request_id': 'req-455a9888-b1e8-4317-8311-c2d85a75421a'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:13.960642 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-455a9888-b1e8-4317-8311-c2d85a75421a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eaf5c3022d449748376965ad6ab87f2', 'request_id': 'req-455a9888-b1e8-4317-8311-c2d85a75421a'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:14.036397 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fcbbb1-5c06-4f14-bfd5-6f1bdd479ec2 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:14.090938 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-455a9888-b1e8-4317-8311-c2d85a75421a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eaf5c3022d449748376965ad6ab87f2', 'request_id': 'req-455a9888-b1e8-4317-8311-c2d85a75421a'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:14.142664 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-455a9888-b1e8-4317-8311-c2d85a75421a admin admin] The token cache is being invalidate because project 5eaf5c3022d449748376965ad6ab87f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:13:14.145814 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-455a9888-b1e8-4317-8311-c2d85a75421a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:14.147643 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-455a9888-b1e8-4317-8311-c2d85a75421a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:14.154649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 881/1750] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:13:13 2022] DELETE /identity/v3/projects/5eaf5c3022d449748376965ad6ab87f2 => generated 0 bytes in 751 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:14.157471 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9b8d4-cdbc-411b-88ca-0accc6ea574d None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:14.159791 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9b8d4-cdbc-411b-88ca-0accc6ea574d None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:14.162210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9b8d4-cdbc-411b-88ca-0accc6ea574d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:14.259639 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-bca9b8d4-cdbc-411b-88ca-0accc6ea574d None None] MFA Rules not processed for user `7b4c766f29e045a284f775220b58cda3`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:14.261019 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-bca9b8d4-cdbc-411b-88ca-0accc6ea574d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:14.287521 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a49224-7d68-469d-a058-3934b64bc0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:14.288672 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a49224-7d68-469d-a058-3934b64bc0dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:14.288672 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a49224-7d68-469d-a058-3934b64bc0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:14.288878 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a49224-7d68-469d-a058-3934b64bc0dc admin admin] PATH_INFO: `/v3/users/76f686ceb7ec41f1a4af40f06502575d` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:14.292588 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a49224-7d68-469d-a058-3934b64bc0dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=76f686ceb7ec41f1a4af40f06502575d)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:14.294575 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a49224-7d68-469d-a058-3934b64bc0dc admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:14.467730 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 882/1751] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:14.476745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f2168-eba1-40d5-85e8-550bab04c518 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:14.476745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f2168-eba1-40d5-85e8-550bab04c518 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:14.476745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f2168-eba1-40d5-85e8-550bab04c518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:14.530275 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-ab3f2168-eba1-40d5-85e8-550bab04c518 None None] MFA Rules not processed for user `f01737ba3200470c8635f335d0338ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:14.532069 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-ab3f2168-eba1-40d5-85e8-550bab04c518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:14.619110 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-99a49224-7d68-469d-a058-3934b64bc0dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76f686ceb7ec41f1a4af40f06502575d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:14.671681 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:14.702353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-99a49224-7d68-469d-a058-3934b64bc0dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76f686ceb7ec41f1a4af40f06502575d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:14.709847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-99a49224-7d68-469d-a058-3934b64bc0dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76f686ceb7ec41f1a4af40f06502575d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:14.730446 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-99a49224-7d68-469d-a058-3934b64bc0dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76f686ceb7ec41f1a4af40f06502575d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:14.733305 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 883/1752] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:14.738928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d51ce-491c-4b3f-b0c0-5511ee55fbd3 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:14.741885 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d51ce-491c-4b3f-b0c0-5511ee55fbd3 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:14.741885 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d51ce-491c-4b3f-b0c0-5511ee55fbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:14.744985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-99a49224-7d68-469d-a058-3934b64bc0dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76f686ceb7ec41f1a4af40f06502575d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:14.781736 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 870/1753] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:13:13 2022] DELETE /identity/v3/users/76f686ceb7ec41f1a4af40f06502575d => generated 0 bytes in 940 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:14.783431 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a49224-7d68-469d-a058-3934b64bc0dc admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:14.786858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-99a49224-7d68-469d-a058-3934b64bc0dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:14.804905 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-b86d51ce-491c-4b3f-b0c0-5511ee55fbd3 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:14.807131 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-b86d51ce-491c-4b3f-b0c0-5511ee55fbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:14.809676 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:15.038644 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 884/1754] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:15.050903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417ff4d-b490-4179-8fed-4db5ba3cfa34 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:15.052857 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417ff4d-b490-4179-8fed-4db5ba3cfa34 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:15.053359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417ff4d-b490-4179-8fed-4db5ba3cfa34 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:15.058983 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 885/1755] 10.222.0.40 () {54 vars in 921 bytes} [Wed Aug 3 05:13:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:13:15.060610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe0535-91ca-41d0-9689-b9c305e8a0f1 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:15.064199 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe0535-91ca-41d0-9689-b9c305e8a0f1 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:15.064948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe0535-91ca-41d0-9689-b9c305e8a0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:15.083615 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a49224-7d68-469d-a058-3934b64bc0dc admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:15.102082 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323af8d-a47c-46dc-af8b-85c679628574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:15.102776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323af8d-a47c-46dc-af8b-85c679628574 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:15.103054 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323af8d-a47c-46dc-af8b-85c679628574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:15.103330 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323af8d-a47c-46dc-af8b-85c679628574 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:15.104018 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4323af8d-a47c-46dc-af8b-85c679628574 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:15.116035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4323af8d-a47c-46dc-af8b-85c679628574 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:15.118287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-3fbe0535-91ca-41d0-9689-b9c305e8a0f1 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:15.122769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-3fbe0535-91ca-41d0-9689-b9c305e8a0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:15.135176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:15.151067 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 886/1756] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:15.153375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe0535-91ca-41d0-9689-b9c305e8a0f1 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:15.181370 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe0535-91ca-41d0-9689-b9c305e8a0f1 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:15.203614 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017511f4-feed-45be-920a-10472702d3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:15.204582 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017511f4-feed-45be-920a-10472702d3f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:15.204980 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017511f4-feed-45be-920a-10472702d3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:15.205422 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017511f4-feed-45be-920a-10472702d3f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:15.206319 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017511f4-feed-45be-920a-10472702d3f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:13:15.206739 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017511f4-feed-45be-920a-10472702d3f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:15.210701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017511f4-feed-45be-920a-10472702d3f5 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:15.246834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 887/1757] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:13:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:13:15.272327 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90fed6b-6c80-4300-871e-eda0778d8699 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:15.276452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90fed6b-6c80-4300-871e-eda0778d8699 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:15.276981 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90fed6b-6c80-4300-871e-eda0778d8699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:15.306295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 871/1758] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:13:14 2022] POST /identity/v3/projects => generated 365 bytes in 523 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:15.308150 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323af8d-a47c-46dc-af8b-85c679628574 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:15.346621 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-b90fed6b-6c80-4300-871e-eda0778d8699 None None] MFA Rules not processed for user `a3fa4a6c16274c18849869b55b33ac62`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:15.349356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-b90fed6b-6c80-4300-871e-eda0778d8699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:15.527824 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 888/1759] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Aug 3 05:13:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:15.528901 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90fed6b-6c80-4300-871e-eda0778d8699 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:15.545450 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323af8d-a47c-46dc-af8b-85c679628574 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:15.554394 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90fed6b-6c80-4300-871e-eda0778d8699 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:15.561880 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707a762-a273-460e-a5d5-faa5fa72cd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:15.562746 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707a762-a273-460e-a5d5-faa5fa72cd8d admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:15.563200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707a762-a273-460e-a5d5-faa5fa72cd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:15.563654 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707a762-a273-460e-a5d5-faa5fa72cd8d admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:15.564591 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4707a762-a273-460e-a5d5-faa5fa72cd8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:15.564930 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d6186-5790-4d1b-aac9-ec1bdb87fb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:15.569360 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d6186-5790-4d1b-aac9-ec1bdb87fb52 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:15.569629 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d6186-5790-4d1b-aac9-ec1bdb87fb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:15.569916 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d6186-5790-4d1b-aac9-ec1bdb87fb52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:15.570618 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d6186-5790-4d1b-aac9-ec1bdb87fb52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:15.571411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4707a762-a273-460e-a5d5-faa5fa72cd8d admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:15.582773 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d6186-5790-4d1b-aac9-ec1bdb87fb52 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:15.678200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 889/1760] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:13:15 2022] POST /identity/v3/users => generated 289 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:15.680079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707a762-a273-460e-a5d5-faa5fa72cd8d admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:15.711625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707a762-a273-460e-a5d5-faa5fa72cd8d admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:15.723723 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb4e09-1802-4e03-bc05-7ed466828d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:15.724934 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb4e09-1802-4e03-bc05-7ed466828d4a admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:15.725381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb4e09-1802-4e03-bc05-7ed466828d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:15.725807 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb4e09-1802-4e03-bc05-7ed466828d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:15.726666 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb4e09-1802-4e03-bc05-7ed466828d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:15.728500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb4e09-1802-4e03-bc05-7ed466828d4a admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:15.761177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 890/1761] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:15.762478 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb4e09-1802-4e03-bc05-7ed466828d4a admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:15.781633 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb4e09-1802-4e03-bc05-7ed466828d4a admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:15.787104 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 872/1762] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:13:15 2022] POST /identity/v3/projects => generated 365 bytes in 476 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:15.790258 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d6186-5790-4d1b-aac9-ec1bdb87fb52 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:16.007801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa760a1e-4122-4d13-8125-f1f82a90c11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:16.022367 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa760a1e-4122-4d13-8125-f1f82a90c11c admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:16.023394 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa760a1e-4122-4d13-8125-f1f82a90c11c admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:16.024655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa760a1e-4122-4d13-8125-f1f82a90c11c admin admin] PATH_INFO: `/v3/projects/f860d7f3a8504bd7b2c36068a020489b/users/9fb3ad1adbbb4c8985a71b79b080b753/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:16.028573 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:16.033648 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d6186-5790-4d1b-aac9-ec1bdb87fb52 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:16.055983 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b051f-abf0-448d-8874-efa0558b92c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:16.056640 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b051f-abf0-448d-8874-efa0558b92c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:16.057043 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b051f-abf0-448d-8874-efa0558b92c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:16.057352 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b051f-abf0-448d-8874-efa0558b92c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:16.058039 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b051f-abf0-448d-8874-efa0558b92c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:16.060164 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b051f-abf0-448d-8874-efa0558b92c6 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:16.064584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa760a1e-4122-4d13-8125-f1f82a90c11c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f860d7f3a8504bd7b2c36068a020489b, user_id=9fb3ad1adbbb4c8985a71b79b080b753, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:16.074329 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa760a1e-4122-4d13-8125-f1f82a90c11c admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:16.196043 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 891/1763] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:15 2022] PUT /identity/v3/projects/f860d7f3a8504bd7b2c36068a020489b/users/9fb3ad1adbbb4c8985a71b79b080b753/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:16.200314 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa760a1e-4122-4d13-8125-f1f82a90c11c admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:16.243300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:16.251952 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 873/1764] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:13:15 2022] POST /identity/v3/users => generated 283 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:16.256766 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b051f-abf0-448d-8874-efa0558b92c6 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:16.278247 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b051f-abf0-448d-8874-efa0558b92c6 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:16.393631 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa760a1e-4122-4d13-8125-f1f82a90c11c admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:16.394716 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1eee15-d3fe-4b4f-8f6e-7e086dde2e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:16.395575 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1eee15-d3fe-4b4f-8f6e-7e086dde2e41 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:16.395886 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1eee15-d3fe-4b4f-8f6e-7e086dde2e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:16.396166 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1eee15-d3fe-4b4f-8f6e-7e086dde2e41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:16.397242 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1eee15-d3fe-4b4f-8f6e-7e086dde2e41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:16.400201 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1eee15-d3fe-4b4f-8f6e-7e086dde2e41 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:16.419448 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 892/1765] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:13:16.420605 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1eee15-d3fe-4b4f-8f6e-7e086dde2e41 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:16.423358 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-cf1eee15-d3fe-4b4f-8f6e-7e086dde2e41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:16.445209 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1eee15-d3fe-4b4f-8f6e-7e086dde2e41 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:16.455251 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d600f59e-567a-44df-862f-718b40b9fa42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:16.456236 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600f59e-567a-44df-862f-718b40b9fa42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:16.456689 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600f59e-567a-44df-862f-718b40b9fa42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:16.457198 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600f59e-567a-44df-862f-718b40b9fa42 admin admin] PATH_INFO: `/v3/projects/28d20d96eda048d888fa884535e48a0c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:16.458469 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d600f59e-567a-44df-862f-718b40b9fa42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28d20d96eda048d888fa884535e48a0c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:16.460751 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d600f59e-567a-44df-862f-718b40b9fa42 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:16.508179 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eadcd2c-80cc-4b75-9ea6-333ff1aa26f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:16.521145 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eadcd2c-80cc-4b75-9ea6-333ff1aa26f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:16.521712 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eadcd2c-80cc-4b75-9ea6-333ff1aa26f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:16.522317 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eadcd2c-80cc-4b75-9ea6-333ff1aa26f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:16.524040 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eadcd2c-80cc-4b75-9ea6-333ff1aa26f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:16.527837 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eadcd2c-80cc-4b75-9ea6-333ff1aa26f4 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:16.634457 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 874/1766] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:13:16.634710 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-d600f59e-567a-44df-862f-718b40b9fa42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28d20d96eda048d888fa884535e48a0c', 'request_id': 'req-d600f59e-567a-44df-862f-718b40b9fa42'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:16.638734 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eadcd2c-80cc-4b75-9ea6-333ff1aa26f4 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:16.675613 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-d600f59e-567a-44df-862f-718b40b9fa42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28d20d96eda048d888fa884535e48a0c', 'request_id': 'req-d600f59e-567a-44df-862f-718b40b9fa42'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:16.696655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eadcd2c-80cc-4b75-9ea6-333ff1aa26f4 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:16.766159 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-d600f59e-567a-44df-862f-718b40b9fa42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28d20d96eda048d888fa884535e48a0c', 'request_id': 'req-d600f59e-567a-44df-862f-718b40b9fa42'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:16.783200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-d600f59e-567a-44df-862f-718b40b9fa42 admin admin] The token cache is being invalidate because project 28d20d96eda048d888fa884535e48a0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:13:16.783492 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-d600f59e-567a-44df-862f-718b40b9fa42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:16.786362 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-d600f59e-567a-44df-862f-718b40b9fa42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:16.788638 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 893/1767] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:13:16 2022] DELETE /identity/v3/projects/28d20d96eda048d888fa884535e48a0c => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:16.792196 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d600f59e-567a-44df-862f-718b40b9fa42 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:16.793687 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-d600f59e-567a-44df-862f-718b40b9fa42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:16.855624 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8476d0-e613-4e39-bc35-8239566ab9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:16.856353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8476d0-e613-4e39-bc35-8239566ab9c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:16.860585 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8476d0-e613-4e39-bc35-8239566ab9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:16.862368 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8476d0-e613-4e39-bc35-8239566ab9c7 admin admin] PATH_INFO: `/v3/projects/f860d7f3a8504bd7b2c36068a020489b/users/9fb3ad1adbbb4c8985a71b79b080b753/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:16.871856 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8476d0-e613-4e39-bc35-8239566ab9c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f860d7f3a8504bd7b2c36068a020489b, user_id=9fb3ad1adbbb4c8985a71b79b080b753, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:16.882924 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8476d0-e613-4e39-bc35-8239566ab9c7 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:16.928104 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 875/1768] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:16 2022] PUT /identity/v3/projects/f860d7f3a8504bd7b2c36068a020489b/users/9fb3ad1adbbb4c8985a71b79b080b753/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:16.929140 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8476d0-e613-4e39-bc35-8239566ab9c7 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:16.942171 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7a8476d0-e613-4e39-bc35-8239566ab9c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:16.960726 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d600f59e-567a-44df-862f-718b40b9fa42 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:17.118340 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d3b3c-63f5-4290-866e-74d48e0196a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:17.118340 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d3b3c-63f5-4290-866e-74d48e0196a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:17.118340 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d3b3c-63f5-4290-866e-74d48e0196a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:17.118340 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d3b3c-63f5-4290-866e-74d48e0196a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:17.119311 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d3b3c-63f5-4290-866e-74d48e0196a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:17.119311 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d3b3c-63f5-4290-866e-74d48e0196a3 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:17.124127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 894/1769] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:17.132685 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2cf20-6409-4243-a5a0-7d1978473466 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:17.135007 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2cf20-6409-4243-a5a0-7d1978473466 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:17.135293 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2cf20-6409-4243-a5a0-7d1978473466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:17.141839 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8476d0-e613-4e39-bc35-8239566ab9c7 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:17.170868 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a9df1-37a4-478a-b027-f33aae7d8f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:17.173384 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a9df1-37a4-478a-b027-f33aae7d8f4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:17.173747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a9df1-37a4-478a-b027-f33aae7d8f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:17.174041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a9df1-37a4-478a-b027-f33aae7d8f4a admin admin] PATH_INFO: `/v3/users/8da1bae26adb4d2c9ce009ae19f96639` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:17.178036 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3a9df1-37a4-478a-b027-f33aae7d8f4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8da1bae26adb4d2c9ce009ae19f96639)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:17.181583 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3a9df1-37a4-478a-b027-f33aae7d8f4a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:17.215739 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-ccf2cf20-6409-4243-a5a0-7d1978473466 None None] MFA Rules not processed for user `9fb3ad1adbbb4c8985a71b79b080b753`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:17.217347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-ccf2cf20-6409-4243-a5a0-7d1978473466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:17.359566 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:INSERT=3,keystone:DELETE=12,keystone:SELECT=85 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:17.376907 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 895/1770] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:17 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:17.378078 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2cf20-6409-4243-a5a0-7d1978473466 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:17.406396 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2cf20-6409-4243-a5a0-7d1978473466 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:17.413933 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beefe44-3acf-4fec-a799-0de367447a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:17.414671 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beefe44-3acf-4fec-a799-0de367447a51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:17.415073 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beefe44-3acf-4fec-a799-0de367447a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:17.415531 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beefe44-3acf-4fec-a799-0de367447a51 admin admin] PATH_INFO: `/v3/projects/fd41573fccb242a2b9277ae51d71dc35/users/db9f5172b9d14a0585e611d28d50c09e/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:17.447046 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beefe44-3acf-4fec-a799-0de367447a51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd41573fccb242a2b9277ae51d71dc35, user_id=db9f5172b9d14a0585e611d28d50c09e, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:17.451628 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beefe44-3acf-4fec-a799-0de367447a51 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:17.508533 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-6a3a9df1-37a4-478a-b027-f33aae7d8f4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8da1bae26adb4d2c9ce009ae19f96639'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:17.546954 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 896/1771] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:17 2022] PUT /identity/v3/projects/fd41573fccb242a2b9277ae51d71dc35/users/db9f5172b9d14a0585e611d28d50c09e/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:17.551136 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beefe44-3acf-4fec-a799-0de367447a51 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:17.554496 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-7beefe44-3acf-4fec-a799-0de367447a51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:17.560848 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:17.632382 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-6a3a9df1-37a4-478a-b027-f33aae7d8f4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8da1bae26adb4d2c9ce009ae19f96639'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:17.634087 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-6a3a9df1-37a4-478a-b027-f33aae7d8f4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8da1bae26adb4d2c9ce009ae19f96639'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:17.677251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-6a3a9df1-37a4-478a-b027-f33aae7d8f4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8da1bae26adb4d2c9ce009ae19f96639'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:17.684782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-6a3a9df1-37a4-478a-b027-f33aae7d8f4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8da1bae26adb4d2c9ce009ae19f96639'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:17.718048 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 876/1772] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:13:16 2022] DELETE /identity/v3/users/8da1bae26adb4d2c9ce009ae19f96639 => generated 0 bytes in 789 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:17.723672 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a9df1-37a4-478a-b027-f33aae7d8f4a admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:17.723672 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6a3a9df1-37a4-478a-b027-f33aae7d8f4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:17.735377 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beefe44-3acf-4fec-a799-0de367447a51 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:17.852110 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a9df1-37a4-478a-b027-f33aae7d8f4a admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:17.876736 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02faac4-ac67-4adc-b8bf-b4f3a10011dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:17.883047 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02faac4-ac67-4adc-b8bf-b4f3a10011dd service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:17.883047 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02faac4-ac67-4adc-b8bf-b4f3a10011dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:17.883047 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02faac4-ac67-4adc-b8bf-b4f3a10011dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:17.884792 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d02faac4-ac67-4adc-b8bf-b4f3a10011dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:17.887718 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7754c-bf48-46c2-8d84-0791921eb1ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:17.888665 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7754c-bf48-46c2-8d84-0791921eb1ab service nova] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:17.888807 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7754c-bf48-46c2-8d84-0791921eb1ab service nova] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:17.889691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7754c-bf48-46c2-8d84-0791921eb1ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:17.892035 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-e2a7754c-bf48-46c2-8d84-0791921eb1ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:17.910168 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a7754c-bf48-46c2-8d84-0791921eb1ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:17.916837 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a7754c-bf48-46c2-8d84-0791921eb1ab service nova] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:17.937106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 897/1773] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:17.938050 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7754c-bf48-46c2-8d84-0791921eb1ab service nova] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:17.940448 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-e2a7754c-bf48-46c2-8d84-0791921eb1ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:18.048552 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02faac4-ac67-4adc-b8bf-b4f3a10011dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:18.050099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02faac4-ac67-4adc-b8bf-b4f3a10011dd service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:18.069676 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7754c-bf48-46c2-8d84-0791921eb1ab service nova] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:18.076383 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 877/1774] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:17 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:18.077830 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a0ce3-62d6-48e3-815a-ccf2d5f50fa0 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:18.078107 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a0ce3-62d6-48e3-815a-ccf2d5f50fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:18.078470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a0ce3-62d6-48e3-815a-ccf2d5f50fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:18.104002 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f104a0b-042c-4c12-9849-c67f7fde0607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:18.104677 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f104a0b-042c-4c12-9849-c67f7fde0607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:18.104945 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f104a0b-042c-4c12-9849-c67f7fde0607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:18.108284 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f104a0b-042c-4c12-9849-c67f7fde0607 admin admin] PATH_INFO: `/v3/projects/27b29ad85d3f431994f53598451a7a97` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:18.113018 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f104a0b-042c-4c12-9849-c67f7fde0607 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27b29ad85d3f431994f53598451a7a97)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:18.116383 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-477a0ce3-62d6-48e3-815a-ccf2d5f50fa0 None None] MFA Rules not processed for user `db9f5172b9d14a0585e611d28d50c09e`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:18.117650 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f104a0b-042c-4c12-9849-c67f7fde0607 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:18.118061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-477a0ce3-62d6-48e3-815a-ccf2d5f50fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:18.285010 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 878/1775] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:18.294131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a0ce3-62d6-48e3-815a-ccf2d5f50fa0 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:18.299814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-477a0ce3-62d6-48e3-815a-ccf2d5f50fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:18.319700 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a0ce3-62d6-48e3-815a-ccf2d5f50fa0 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:18.340273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628d542-ac40-4419-8031-97d5b5346591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:18.340854 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628d542-ac40-4419-8031-97d5b5346591 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:18.341206 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628d542-ac40-4419-8031-97d5b5346591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:18.341552 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628d542-ac40-4419-8031-97d5b5346591 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:18.342251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0628d542-ac40-4419-8031-97d5b5346591 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:18.344511 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0628d542-ac40-4419-8031-97d5b5346591 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:18.408709 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-3f104a0b-042c-4c12-9849-c67f7fde0607 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27b29ad85d3f431994f53598451a7a97', 'request_id': 'req-3f104a0b-042c-4c12-9849-c67f7fde0607'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:18.429380 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-3f104a0b-042c-4c12-9849-c67f7fde0607 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27b29ad85d3f431994f53598451a7a97', 'request_id': 'req-3f104a0b-042c-4c12-9849-c67f7fde0607'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:18.484057 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:18.501325 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 879/1776] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:13:18 2022] POST /identity/v3/projects => generated 365 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:18.505249 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628d542-ac40-4419-8031-97d5b5346591 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:18.554027 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-3f104a0b-042c-4c12-9849-c67f7fde0607 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27b29ad85d3f431994f53598451a7a97', 'request_id': 'req-3f104a0b-042c-4c12-9849-c67f7fde0607'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:18.583202 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-3f104a0b-042c-4c12-9849-c67f7fde0607 admin admin] The token cache is being invalidate because project 27b29ad85d3f431994f53598451a7a97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:13:18.584148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-3f104a0b-042c-4c12-9849-c67f7fde0607 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:18.589149 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-3f104a0b-042c-4c12-9849-c67f7fde0607 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:18.599724 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 898/1777] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:13:17 2022] DELETE /identity/v3/projects/27b29ad85d3f431994f53598451a7a97 => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:18.602981 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e27e4-508c-4cd5-809f-1082361cbc32 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:18.603352 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e27e4-508c-4cd5-809f-1082361cbc32 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:18.609865 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e27e4-508c-4cd5-809f-1082361cbc32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:18.628463 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:18.663383 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-1b4e27e4-508c-4cd5-809f-1082361cbc32 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:18.664373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-1b4e27e4-508c-4cd5-809f-1082361cbc32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:18.689848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628d542-ac40-4419-8031-97d5b5346591 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:18.699688 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 899/1778] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:18.701510 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeed255-f708-4acb-be29-d76ec0222525 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:18.701914 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeed255-f708-4acb-be29-d76ec0222525 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:18.702206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeed255-f708-4acb-be29-d76ec0222525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:18.708788 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c48fc2-460c-44b9-a702-95fc1ab914cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:18.709442 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c48fc2-460c-44b9-a702-95fc1ab914cb admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:18.712656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c48fc2-460c-44b9-a702-95fc1ab914cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:18.712656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c48fc2-460c-44b9-a702-95fc1ab914cb admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:18.712656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c48fc2-460c-44b9-a702-95fc1ab914cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:18.712656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c48fc2-460c-44b9-a702-95fc1ab914cb admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:18.735907 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-daeed255-f708-4acb-be29-d76ec0222525 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:18.735907 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-daeed255-f708-4acb-be29-d76ec0222525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:18.758994 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 900/1779] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:13:18.763277 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeed255-f708-4acb-be29-d76ec0222525 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:18.798302 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeed255-f708-4acb-be29-d76ec0222525 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:18.815359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f55859-06f1-4fb9-8d6b-26d4d7ce9d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:18.816282 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f55859-06f1-4fb9-8d6b-26d4d7ce9d96 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:18.816846 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f55859-06f1-4fb9-8d6b-26d4d7ce9d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:18.817134 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f55859-06f1-4fb9-8d6b-26d4d7ce9d96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:18.821162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f55859-06f1-4fb9-8d6b-26d4d7ce9d96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:13:18.824821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f55859-06f1-4fb9-8d6b-26d4d7ce9d96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:18.826025 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f55859-06f1-4fb9-8d6b-26d4d7ce9d96 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:18.862159 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 880/1780] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:13:18 2022] POST /identity/v3/users => generated 289 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:18.863771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c48fc2-460c-44b9-a702-95fc1ab914cb admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:18.871637 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c8c48fc2-460c-44b9-a702-95fc1ab914cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:18.884323 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 901/1781] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:13:18.892273 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f55859-06f1-4fb9-8d6b-26d4d7ce9d96 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:18.929226 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c48fc2-460c-44b9-a702-95fc1ab914cb admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:18.932543 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f55859-06f1-4fb9-8d6b-26d4d7ce9d96 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:18.952216 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2123165-cec2-43a3-9285-c5efda7107af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:18.952981 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2123165-cec2-43a3-9285-c5efda7107af admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:18.953271 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2123165-cec2-43a3-9285-c5efda7107af admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:18.953612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2123165-cec2-43a3-9285-c5efda7107af admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:18.954413 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2123165-cec2-43a3-9285-c5efda7107af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:18.956416 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce43f9-ff36-47e6-b788-883cbbc1d304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:18.957740 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce43f9-ff36-47e6-b788-883cbbc1d304 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:18.958065 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce43f9-ff36-47e6-b788-883cbbc1d304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:18.958337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce43f9-ff36-47e6-b788-883cbbc1d304 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:18.959110 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce43f9-ff36-47e6-b788-883cbbc1d304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:18.961691 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce43f9-ff36-47e6-b788-883cbbc1d304 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:18.963729 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2123165-cec2-43a3-9285-c5efda7107af admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:18.996295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 881/1782] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:18.997433 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce43f9-ff36-47e6-b788-883cbbc1d304 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:19.014566 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce43f9-ff36-47e6-b788-883cbbc1d304 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:19.024213 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b12093f-dbbb-4b67-810f-408f5634f575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:19.024213 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12093f-dbbb-4b67-810f-408f5634f575 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:19.024213 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12093f-dbbb-4b67-810f-408f5634f575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:19.024720 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12093f-dbbb-4b67-810f-408f5634f575 admin admin] PATH_INFO: `/v3/projects/f242d01b08704c189d53de46334308a1/users/c4ca3ca7812e4413bca591321e2e1b2a/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:19.053096 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b12093f-dbbb-4b67-810f-408f5634f575 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f242d01b08704c189d53de46334308a1, user_id=c4ca3ca7812e4413bca591321e2e1b2a, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:19.055300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b12093f-dbbb-4b67-810f-408f5634f575 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:19.085038 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 902/1783] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:13:18 2022] POST /identity/v3/projects => generated 383 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:19.086456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2123165-cec2-43a3-9285-c5efda7107af admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:19.151498 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 882/1784] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:18 2022] PUT /identity/v3/projects/f242d01b08704c189d53de46334308a1/users/c4ca3ca7812e4413bca591321e2e1b2a/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:19.153186 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b12093f-dbbb-4b67-810f-408f5634f575 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:19.264825 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2123165-cec2-43a3-9285-c5efda7107af admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:19.307499 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b12093f-dbbb-4b67-810f-408f5634f575 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:19.313992 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d28f3-43b8-436d-ae4a-d304a05089cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:19.315365 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d28f3-43b8-436d-ae4a-d304a05089cf admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:19.315884 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d28f3-43b8-436d-ae4a-d304a05089cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:19.317859 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d28f3-43b8-436d-ae4a-d304a05089cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:19.320137 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d28f3-43b8-436d-ae4a-d304a05089cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:19.322391 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d28f3-43b8-436d-ae4a-d304a05089cf admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:19.358301 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 883/1785] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:19.359696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d28f3-43b8-436d-ae4a-d304a05089cf admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:19.405022 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d28f3-43b8-436d-ae4a-d304a05089cf admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:19.413259 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decab400-dc83-4cc8-b136-3311736c9125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:19.414950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decab400-dc83-4cc8-b136-3311736c9125 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:19.415876 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decab400-dc83-4cc8-b136-3311736c9125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:19.416782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decab400-dc83-4cc8-b136-3311736c9125 admin admin] PATH_INFO: `/v3/projects/f242d01b08704c189d53de46334308a1/users/c4ca3ca7812e4413bca591321e2e1b2a/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:19.421767 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decab400-dc83-4cc8-b136-3311736c9125 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f242d01b08704c189d53de46334308a1, user_id=c4ca3ca7812e4413bca591321e2e1b2a, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:19.425450 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decab400-dc83-4cc8-b136-3311736c9125 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:19.455346 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 884/1786] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:19 2022] PUT /identity/v3/projects/f242d01b08704c189d53de46334308a1/users/c4ca3ca7812e4413bca591321e2e1b2a/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:19.457340 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709ec61-a949-40a3-8be2-2e6277847a4f None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:19.457640 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709ec61-a949-40a3-8be2-2e6277847a4f None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:19.457918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709ec61-a949-40a3-8be2-2e6277847a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:19.479576 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ab7d9-ad19-451c-9011-c393a76a5869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:19.480914 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ab7d9-ad19-451c-9011-c393a76a5869 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:19.482927 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ab7d9-ad19-451c-9011-c393a76a5869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:19.483255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ab7d9-ad19-451c-9011-c393a76a5869 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:19.487046 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ab7d9-ad19-451c-9011-c393a76a5869 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:19.490323 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ab7d9-ad19-451c-9011-c393a76a5869 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:19.515729 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-0709ec61-a949-40a3-8be2-2e6277847a4f None None] MFA Rules not processed for user `c4ca3ca7812e4413bca591321e2e1b2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:19.525750 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0709ec61-a949-40a3-8be2-2e6277847a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:19.672511 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:19.715408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:19.725198 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 885/1787] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:19 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:19.742304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 903/1788] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:13:19 2022] POST /identity/v3/users => generated 292 bytes in 657 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:19.753570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709ec61-a949-40a3-8be2-2e6277847a4f None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:19.754888 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ab7d9-ad19-451c-9011-c393a76a5869 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:19.758810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0709ec61-a949-40a3-8be2-2e6277847a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:19.784781 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ab7d9-ad19-451c-9011-c393a76a5869 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:19.965379 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709ec61-a949-40a3-8be2-2e6277847a4f None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:19.971332 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720dd8d-23cd-4e67-932f-d29dd24cb24c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:19.971856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58270188-fdbf-4c37-9f2c-67b3ab0a0576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:19.973080 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58270188-fdbf-4c37-9f2c-67b3ab0a0576 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:19.973466 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58270188-fdbf-4c37-9f2c-67b3ab0a0576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:19.974097 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58270188-fdbf-4c37-9f2c-67b3ab0a0576 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:19.974307 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720dd8d-23cd-4e67-932f-d29dd24cb24c service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:19.974734 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720dd8d-23cd-4e67-932f-d29dd24cb24c service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:19.975178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720dd8d-23cd-4e67-932f-d29dd24cb24c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:19.975356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58270188-fdbf-4c37-9f2c-67b3ab0a0576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:19.977651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58270188-fdbf-4c37-9f2c-67b3ab0a0576 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:19.993317 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 904/1789] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:19.994409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58270188-fdbf-4c37-9f2c-67b3ab0a0576 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:19.996644 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-58270188-fdbf-4c37-9f2c-67b3ab0a0576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:20.010543 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720dd8d-23cd-4e67-932f-d29dd24cb24c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:20.012852 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720dd8d-23cd-4e67-932f-d29dd24cb24c service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:20.030496 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58270188-fdbf-4c37-9f2c-67b3ab0a0576 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:20.036732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b2b9c-ea64-4f1d-804c-4c91fde7beb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:20.037480 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b2b9c-ea64-4f1d-804c-4c91fde7beb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:20.037805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b2b9c-ea64-4f1d-804c-4c91fde7beb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:20.038096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b2b9c-ea64-4f1d-804c-4c91fde7beb4 admin admin] PATH_INFO: `/v3/users/7b4c766f29e045a284f775220b58cda3` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:20.039649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b2b9c-ea64-4f1d-804c-4c91fde7beb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b4c766f29e045a284f775220b58cda3)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:20.041714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 886/1790] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:19 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:20.042756 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720dd8d-23cd-4e67-932f-d29dd24cb24c service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:20.046538 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b2b9c-ea64-4f1d-804c-4c91fde7beb4 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:20.073494 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720dd8d-23cd-4e67-932f-d29dd24cb24c service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:20.084793 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837177e3-7148-47f2-9068-4d717b8daeb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:20.085436 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837177e3-7148-47f2-9068-4d717b8daeb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:20.085715 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837177e3-7148-47f2-9068-4d717b8daeb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:20.085988 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837177e3-7148-47f2-9068-4d717b8daeb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:20.086684 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837177e3-7148-47f2-9068-4d717b8daeb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:20.088318 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837177e3-7148-47f2-9068-4d717b8daeb0 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:20.114881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 887/1791] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:20.119308 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837177e3-7148-47f2-9068-4d717b8daeb0 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:20.136914 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837177e3-7148-47f2-9068-4d717b8daeb0 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:20.142318 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b697202-23aa-4264-918c-b24257ce1589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:20.143306 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b697202-23aa-4264-918c-b24257ce1589 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:20.144216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b697202-23aa-4264-918c-b24257ce1589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:20.145223 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b697202-23aa-4264-918c-b24257ce1589 admin admin] PATH_INFO: `/v3/projects/fd87dd9e00554bc68e512faeb5ee0de4/users/17707aa716d5442797536630b6e73112/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:20.164728 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b697202-23aa-4264-918c-b24257ce1589 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd87dd9e00554bc68e512faeb5ee0de4, user_id=17707aa716d5442797536630b6e73112, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:20.168239 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b697202-23aa-4264-918c-b24257ce1589 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:20.283129 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 888/1792] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:20 2022] PUT /identity/v3/projects/fd87dd9e00554bc68e512faeb5ee0de4/users/17707aa716d5442797536630b6e73112/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:13:20.289300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:20.295904 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852d842-4761-42b6-91b2-c29f1cc88a7b None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:20.296520 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852d842-4761-42b6-91b2-c29f1cc88a7b None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:20.297010 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852d842-4761-42b6-91b2-c29f1cc88a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:20.345771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-e852d842-4761-42b6-91b2-c29f1cc88a7b None None] MFA Rules not processed for user `17707aa716d5442797536630b6e73112`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:20.347455 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e852d842-4761-42b6-91b2-c29f1cc88a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:20.391095 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a02b2b9c-ea64-4f1d-804c-4c91fde7beb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b4c766f29e045a284f775220b58cda3'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:20.469120 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 889/1793] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:20.471235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852d842-4761-42b6-91b2-c29f1cc88a7b None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:20.508683 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a02b2b9c-ea64-4f1d-804c-4c91fde7beb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b4c766f29e045a284f775220b58cda3'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:20.511312 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a02b2b9c-ea64-4f1d-804c-4c91fde7beb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b4c766f29e045a284f775220b58cda3'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:20.554070 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a02b2b9c-ea64-4f1d-804c-4c91fde7beb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b4c766f29e045a284f775220b58cda3'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:20.555271 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a02b2b9c-ea64-4f1d-804c-4c91fde7beb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b4c766f29e045a284f775220b58cda3'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:20.584296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 905/1794] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:13:19 2022] DELETE /identity/v3/users/7b4c766f29e045a284f775220b58cda3 => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:20.584296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b2b9c-ea64-4f1d-804c-4c91fde7beb4 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:20.584296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-a02b2b9c-ea64-4f1d-804c-4c91fde7beb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:20.646737 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852d842-4761-42b6-91b2-c29f1cc88a7b None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:20.719390 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b2b9c-ea64-4f1d-804c-4c91fde7beb4 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:20.748082 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51030aff-c57a-4632-adbb-2ce53125b64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:20.748837 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51030aff-c57a-4632-adbb-2ce53125b64f admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:20.749339 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51030aff-c57a-4632-adbb-2ce53125b64f admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:20.749669 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51030aff-c57a-4632-adbb-2ce53125b64f admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:20.750401 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51030aff-c57a-4632-adbb-2ce53125b64f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:20.752970 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51030aff-c57a-4632-adbb-2ce53125b64f admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:20.754675 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed71ff-207f-4afb-a326-0d90ea42b6ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:20.755432 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed71ff-207f-4afb-a326-0d90ea42b6ea service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:20.755726 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed71ff-207f-4afb-a326-0d90ea42b6ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:20.755991 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed71ff-207f-4afb-a326-0d90ea42b6ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:20.757983 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3bed71ff-207f-4afb-a326-0d90ea42b6ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:20.840794 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:20.850453 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 906/1795] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:13:20 2022] POST /identity/v3/projects => generated 381 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:20.851447 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51030aff-c57a-4632-adbb-2ce53125b64f admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:20.853124 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-51030aff-c57a-4632-adbb-2ce53125b64f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:20.866531 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed71ff-207f-4afb-a326-0d90ea42b6ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:20.868090 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed71ff-207f-4afb-a326-0d90ea42b6ea service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:20.879993 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 890/1796] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:20.881322 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450d143-d6fe-4815-bac7-0e59b02fd36b None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:20.881764 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450d143-d6fe-4815-bac7-0e59b02fd36b None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:20.882270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450d143-d6fe-4815-bac7-0e59b02fd36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:20.907935 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-9450d143-d6fe-4815-bac7-0e59b02fd36b None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:20.909391 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9450d143-d6fe-4815-bac7-0e59b02fd36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:20.951709 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51030aff-c57a-4632-adbb-2ce53125b64f admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:20.958348 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05503bd-acee-4338-b8ad-73b3e50152c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:20.959096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05503bd-acee-4338-b8ad-73b3e50152c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:20.959522 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05503bd-acee-4338-b8ad-73b3e50152c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:20.960119 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05503bd-acee-4338-b8ad-73b3e50152c3 admin admin] PATH_INFO: `/v3/projects/23c74638205446c888bd8273c25bb103` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:20.961666 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05503bd-acee-4338-b8ad-73b3e50152c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23c74638205446c888bd8273c25bb103)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:20.965473 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05503bd-acee-4338-b8ad-73b3e50152c3 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:21.028247 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 891/1797] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:21.029253 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450d143-d6fe-4815-bac7-0e59b02fd36b None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:21.043543 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450d143-d6fe-4815-bac7-0e59b02fd36b None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:21.056279 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b70c2-68ef-4408-96d0-719e6b7d2e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:21.057000 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b70c2-68ef-4408-96d0-719e6b7d2e45 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:21.057370 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b70c2-68ef-4408-96d0-719e6b7d2e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:21.057652 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b70c2-68ef-4408-96d0-719e6b7d2e45 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:21.058762 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b70c2-68ef-4408-96d0-719e6b7d2e45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:21.061600 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b70c2-68ef-4408-96d0-719e6b7d2e45 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:21.212011 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-f05503bd-acee-4338-b8ad-73b3e50152c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23c74638205446c888bd8273c25bb103', 'request_id': 'req-f05503bd-acee-4338-b8ad-73b3e50152c3'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:21.220206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-f05503bd-acee-4338-b8ad-73b3e50152c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23c74638205446c888bd8273c25bb103', 'request_id': 'req-f05503bd-acee-4338-b8ad-73b3e50152c3'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:21.261127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 892/1798] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:13:21 2022] POST /identity/v3/users => generated 297 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:21.262400 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b70c2-68ef-4408-96d0-719e6b7d2e45 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:21.266824 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0e3b70c2-68ef-4408-96d0-719e6b7d2e45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:21.323316 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-f05503bd-acee-4338-b8ad-73b3e50152c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23c74638205446c888bd8273c25bb103', 'request_id': 'req-f05503bd-acee-4338-b8ad-73b3e50152c3'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:21.345965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-f05503bd-acee-4338-b8ad-73b3e50152c3 admin admin] The token cache is being invalidate because project 23c74638205446c888bd8273c25bb103 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:13:21.346440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-f05503bd-acee-4338-b8ad-73b3e50152c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:21.350191 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-f05503bd-acee-4338-b8ad-73b3e50152c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:21.363921 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 907/1799] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:13:20 2022] DELETE /identity/v3/projects/23c74638205446c888bd8273c25bb103 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:21.364816 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05503bd-acee-4338-b8ad-73b3e50152c3 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:21.369897 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-f05503bd-acee-4338-b8ad-73b3e50152c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:21.476633 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b70c2-68ef-4408-96d0-719e6b7d2e45 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:21.545022 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05503bd-acee-4338-b8ad-73b3e50152c3 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:21.558498 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a31843-4dde-464a-9c61-1aee385bddec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:21.560111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a31843-4dde-464a-9c61-1aee385bddec admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:21.560844 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a31843-4dde-464a-9c61-1aee385bddec admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:21.561194 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a31843-4dde-464a-9c61-1aee385bddec admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:21.563493 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a31843-4dde-464a-9c61-1aee385bddec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:21.565194 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a31843-4dde-464a-9c61-1aee385bddec admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:21.576941 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:21.604458 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 908/1800] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:21.616058 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005ad69-1408-4f73-bd62-e5f39875e1af None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:21.616058 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005ad69-1408-4f73-bd62-e5f39875e1af None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:21.616058 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005ad69-1408-4f73-bd62-e5f39875e1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:21.648509 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f57c41-027d-4565-a332-fed321ac1fb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:21.649441 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f57c41-027d-4565-a332-fed321ac1fb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:21.650527 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f57c41-027d-4565-a332-fed321ac1fb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:21.651013 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f57c41-027d-4565-a332-fed321ac1fb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:21.654092 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f1f57c41-027d-4565-a332-fed321ac1fb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:21.670584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-a005ad69-1408-4f73-bd62-e5f39875e1af None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:21.672014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-a005ad69-1408-4f73-bd62-e5f39875e1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:21.685411 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f57c41-027d-4565-a332-fed321ac1fb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:21.687351 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f57c41-027d-4565-a332-fed321ac1fb1 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:21.688054 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 909/1801] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:21.691739 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005ad69-1408-4f73-bd62-e5f39875e1af None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:21.714174 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005ad69-1408-4f73-bd62-e5f39875e1af None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:21.717122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 893/1802] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:21.719855 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd1082-1603-4bc7-9c88-f09c48b1c4f7 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:21.720791 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd1082-1603-4bc7-9c88-f09c48b1c4f7 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:21.721237 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd1082-1603-4bc7-9c88-f09c48b1c4f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:21.727905 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe71f2-1cc2-4114-b23b-08ef2c598072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:21.730100 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe71f2-1cc2-4114-b23b-08ef2c598072 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:21.730871 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe71f2-1cc2-4114-b23b-08ef2c598072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:21.733192 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe71f2-1cc2-4114-b23b-08ef2c598072 admin admin] PATH_INFO: `/v3/projects/fbc9dc1e38df41a39bac79e2ce3d212e/users/ee80bae4563543e58872ce0ed1f08044/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:21.768629 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe71f2-1cc2-4114-b23b-08ef2c598072 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbc9dc1e38df41a39bac79e2ce3d212e, user_id=ee80bae4563543e58872ce0ed1f08044, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:21.775618 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe71f2-1cc2-4114-b23b-08ef2c598072 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:21.778126 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-4fbd1082-1603-4bc7-9c88-f09c48b1c4f7 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:21.780588 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-4fbd1082-1603-4bc7-9c88-f09c48b1c4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:21.805623 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 894/1803] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:21.810238 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd1082-1603-4bc7-9c88-f09c48b1c4f7 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:21.825677 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd1082-1603-4bc7-9c88-f09c48b1c4f7 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:21.832691 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34939039-7b63-4d96-b324-8f95a3df9f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:21.833355 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34939039-7b63-4d96-b324-8f95a3df9f49 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:21.833636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34939039-7b63-4d96-b324-8f95a3df9f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:21.833914 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34939039-7b63-4d96-b324-8f95a3df9f49 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:21.834706 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34939039-7b63-4d96-b324-8f95a3df9f49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:13:21.835169 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34939039-7b63-4d96-b324-8f95a3df9f49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:21.836843 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34939039-7b63-4d96-b324-8f95a3df9f49 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:21.851845 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 895/1804] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:13:21.859988 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34939039-7b63-4d96-b324-8f95a3df9f49 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:21.869310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 910/1805] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:21 2022] PUT /identity/v3/projects/fbc9dc1e38df41a39bac79e2ce3d212e/users/ee80bae4563543e58872ce0ed1f08044/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:21.873223 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe71f2-1cc2-4114-b23b-08ef2c598072 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:21.892438 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34939039-7b63-4d96-b324-8f95a3df9f49 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:21.989450 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2d5a6-01b0-46ed-81c7-a62c14b15205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:21.991321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2d5a6-01b0-46ed-81c7-a62c14b15205 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:21.991321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2d5a6-01b0-46ed-81c7-a62c14b15205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:21.991321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2d5a6-01b0-46ed-81c7-a62c14b15205 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:21.991880 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c2d5a6-01b0-46ed-81c7-a62c14b15205 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:21.994042 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c2d5a6-01b0-46ed-81c7-a62c14b15205 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:22.059521 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe71f2-1cc2-4114-b23b-08ef2c598072 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:22.069764 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011aeaf-d355-48ff-a67e-c829c9f87744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:22.073301 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011aeaf-d355-48ff-a67e-c829c9f87744 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:22.073632 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011aeaf-d355-48ff-a67e-c829c9f87744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:22.073916 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011aeaf-d355-48ff-a67e-c829c9f87744 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:22.074888 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a011aeaf-d355-48ff-a67e-c829c9f87744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:22.078092 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a011aeaf-d355-48ff-a67e-c829c9f87744 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:22.111937 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 896/1806] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:13:21 2022] POST /identity/v3/projects => generated 387 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:22.114801 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2d5a6-01b0-46ed-81c7-a62c14b15205 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:22.117106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 911/1807] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:22.118107 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011aeaf-d355-48ff-a67e-c829c9f87744 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:22.146440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011aeaf-d355-48ff-a67e-c829c9f87744 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:22.263492 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e87e2-fee2-4999-b540-d02f51f93583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:22.264439 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e87e2-fee2-4999-b540-d02f51f93583 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:22.264710 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e87e2-fee2-4999-b540-d02f51f93583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:22.264898 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e87e2-fee2-4999-b540-d02f51f93583 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:22.265581 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e87e2-fee2-4999-b540-d02f51f93583 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:22.282970 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e87e2-fee2-4999-b540-d02f51f93583 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:22.326206 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2d5a6-01b0-46ed-81c7-a62c14b15205 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:22.343754 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc4fd2-f9ac-4bfe-877b-17f83cb1e74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:22.345015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc4fd2-f9ac-4bfe-877b-17f83cb1e74d admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:22.345681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc4fd2-f9ac-4bfe-877b-17f83cb1e74d admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:22.346177 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc4fd2-f9ac-4bfe-877b-17f83cb1e74d admin admin] PATH_INFO: `/v3/projects/fbc9dc1e38df41a39bac79e2ce3d212e/users/ee80bae4563543e58872ce0ed1f08044/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:22.349568 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdc4fd2-f9ac-4bfe-877b-17f83cb1e74d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbc9dc1e38df41a39bac79e2ce3d212e, user_id=ee80bae4563543e58872ce0ed1f08044, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:22.353262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdc4fd2-f9ac-4bfe-877b-17f83cb1e74d admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:22.372049 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 897/1808] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:22 2022] PUT /identity/v3/projects/fbc9dc1e38df41a39bac79e2ce3d212e/users/ee80bae4563543e58872ce0ed1f08044/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:22.385373 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f75fc3-2498-4b30-bc45-12dfab6d2eb5 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:22.385701 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f75fc3-2498-4b30-bc45-12dfab6d2eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:22.385962 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f75fc3-2498-4b30-bc45-12dfab6d2eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:22.424265 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-19f75fc3-2498-4b30-bc45-12dfab6d2eb5 None None] MFA Rules not processed for user `ee80bae4563543e58872ce0ed1f08044`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:22.425554 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-19f75fc3-2498-4b30-bc45-12dfab6d2eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:22.542213 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 898/1809] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:22 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:13:22.555525 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f75fc3-2498-4b30-bc45-12dfab6d2eb5 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:22.557420 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-19f75fc3-2498-4b30-bc45-12dfab6d2eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:22.625110 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f75fc3-2498-4b30-bc45-12dfab6d2eb5 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:22.630379 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:22.633472 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc9794e-8d9c-42ae-a39c-a168a1ad0bfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:22.639144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9794e-8d9c-42ae-a39c-a168a1ad0bfd service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:22.639144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9794e-8d9c-42ae-a39c-a168a1ad0bfd service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:22.639144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9794e-8d9c-42ae-a39c-a168a1ad0bfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:22.639395 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 912/1810] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:13:22 2022] POST /identity/v3/users => generated 294 bytes in 521 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:22.639979 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e87e2-fee2-4999-b540-d02f51f93583 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:22.645389 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-d92e87e2-fee2-4999-b540-d02f51f93583 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:22.666816 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc9794e-8d9c-42ae-a39c-a168a1ad0bfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:22.671036 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e87e2-fee2-4999-b540-d02f51f93583 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:22.671782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc9794e-8d9c-42ae-a39c-a168a1ad0bfd service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:22.683873 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 899/1811] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:22 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:13:22.700058 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc9794e-8d9c-42ae-a39c-a168a1ad0bfd service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:22.809910 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc9794e-8d9c-42ae-a39c-a168a1ad0bfd service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:22.811428 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c09211-1d41-467d-889e-34ce84cac86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:22.812319 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c09211-1d41-467d-889e-34ce84cac86d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:22.812816 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c09211-1d41-467d-889e-34ce84cac86d admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:22.814253 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c09211-1d41-467d-889e-34ce84cac86d admin admin] PATH_INFO: `/v3/projects/7f816875cdf441c08e8578f95c51f8b5` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:22.815273 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c09211-1d41-467d-889e-34ce84cac86d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f816875cdf441c08e8578f95c51f8b5)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:22.817372 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c09211-1d41-467d-889e-34ce84cac86d admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:22.817688 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f934ed6d-5f20-440b-9683-51fc4bf36063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:22.819093 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f934ed6d-5f20-440b-9683-51fc4bf36063 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:22.819850 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f934ed6d-5f20-440b-9683-51fc4bf36063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:22.821178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f934ed6d-5f20-440b-9683-51fc4bf36063 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:22.821904 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f934ed6d-5f20-440b-9683-51fc4bf36063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:22.823622 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f934ed6d-5f20-440b-9683-51fc4bf36063 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:22.844869 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 900/1812] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:22 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:22.845978 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f934ed6d-5f20-440b-9683-51fc4bf36063 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:22.858483 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f934ed6d-5f20-440b-9683-51fc4bf36063 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:22.864152 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96805e5-e5e7-469e-b459-c11ff5b43829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:22.869005 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96805e5-e5e7-469e-b459-c11ff5b43829 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:22.869005 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96805e5-e5e7-469e-b459-c11ff5b43829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:22.869005 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96805e5-e5e7-469e-b459-c11ff5b43829 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:22.869005 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96805e5-e5e7-469e-b459-c11ff5b43829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:22.869005 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96805e5-e5e7-469e-b459-c11ff5b43829 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:22.889729 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 901/1813] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:22.891398 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96805e5-e5e7-469e-b459-c11ff5b43829 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:22.914924 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96805e5-e5e7-469e-b459-c11ff5b43829 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:22.921552 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7601ac02-6e0b-4bf7-83c1-2ca014e55279 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:22.923054 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7601ac02-6e0b-4bf7-83c1-2ca014e55279 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:22.923401 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7601ac02-6e0b-4bf7-83c1-2ca014e55279 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:22.923683 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7601ac02-6e0b-4bf7-83c1-2ca014e55279 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:22.925703 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7601ac02-6e0b-4bf7-83c1-2ca014e55279 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:22.996800 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-b4c09211-1d41-467d-889e-34ce84cac86d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f816875cdf441c08e8578f95c51f8b5', 'request_id': 'req-b4c09211-1d41-467d-889e-34ce84cac86d'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:23.021405 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7601ac02-6e0b-4bf7-83c1-2ca014e55279 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:23.023302 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7601ac02-6e0b-4bf7-83c1-2ca014e55279 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:23.026165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-b4c09211-1d41-467d-889e-34ce84cac86d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f816875cdf441c08e8578f95c51f8b5', 'request_id': 'req-b4c09211-1d41-467d-889e-34ce84cac86d'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:23.040818 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 902/1814] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:23.042353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7601ac02-6e0b-4bf7-83c1-2ca014e55279 service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:23.057808 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7601ac02-6e0b-4bf7-83c1-2ca014e55279 service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:23.129987 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f24fd51-c5de-4522-a599-dcd334d2b0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:23.130499 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24fd51-c5de-4522-a599-dcd334d2b0b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:23.130778 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24fd51-c5de-4522-a599-dcd334d2b0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:23.131084 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24fd51-c5de-4522-a599-dcd334d2b0b7 admin admin] PATH_INFO: `/v3/projects/7e3d7b457546483381a91a3d9a55a95f/users/807ae4811a134a5f979cda98adaf7ff3/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:23.160337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f24fd51-c5de-4522-a599-dcd334d2b0b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e3d7b457546483381a91a3d9a55a95f, user_id=807ae4811a134a5f979cda98adaf7ff3, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:23.160457 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-b4c09211-1d41-467d-889e-34ce84cac86d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f816875cdf441c08e8578f95c51f8b5', 'request_id': 'req-b4c09211-1d41-467d-889e-34ce84cac86d'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:23.160637 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f24fd51-c5de-4522-a599-dcd334d2b0b7 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:23.166308 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-b4c09211-1d41-467d-889e-34ce84cac86d admin admin] The token cache is being invalidate because project 7f816875cdf441c08e8578f95c51f8b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:13:23.166734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-b4c09211-1d41-467d-889e-34ce84cac86d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:23.168108 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-b4c09211-1d41-467d-889e-34ce84cac86d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:23.171080 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 913/1815] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:13:22 2022] DELETE /identity/v3/projects/7f816875cdf441c08e8578f95c51f8b5 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:23.173998 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c09211-1d41-467d-889e-34ce84cac86d admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:23.175395 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-b4c09211-1d41-467d-889e-34ce84cac86d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:23.192816 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c09211-1d41-467d-889e-34ce84cac86d admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:23.197997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbaa757-16dc-410c-8439-c9644317581c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:23.198734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbaa757-16dc-410c-8439-c9644317581c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:23.199120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbaa757-16dc-410c-8439-c9644317581c admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:23.199488 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbaa757-16dc-410c-8439-c9644317581c admin admin] PATH_INFO: `/v3/users/9fb3ad1adbbb4c8985a71b79b080b753` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:23.200793 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbaa757-16dc-410c-8439-c9644317581c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fb3ad1adbbb4c8985a71b79b080b753)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:23.203002 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbaa757-16dc-410c-8439-c9644317581c admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:23.291514 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 903/1816] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:23 2022] PUT /identity/v3/projects/7e3d7b457546483381a91a3d9a55a95f/users/807ae4811a134a5f979cda98adaf7ff3/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:23.306284 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfe930-9886-4d3d-ac59-41888ce590e2 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:23.306617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfe930-9886-4d3d-ac59-41888ce590e2 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:23.306928 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfe930-9886-4d3d-ac59-41888ce590e2 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:23.312707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 904/1817] 10.222.0.40 () {54 vars in 921 bytes} [Wed Aug 3 05:13:23 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:13:23.313991 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdf1ea-da8b-42e5-8928-9fe6e1ceff9f None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:23.314283 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdf1ea-da8b-42e5-8928-9fe6e1ceff9f None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:23.314560 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdf1ea-da8b-42e5-8928-9fe6e1ceff9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:23.359788 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-5bfdf1ea-da8b-42e5-8928-9fe6e1ceff9f None None] MFA Rules not processed for user `807ae4811a134a5f979cda98adaf7ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:23.361122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-5bfdf1ea-da8b-42e5-8928-9fe6e1ceff9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:23.430834 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 905/1818] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:23.435326 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdf1ea-da8b-42e5-8928-9fe6e1ceff9f None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:23.437304 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-5bfdf1ea-da8b-42e5-8928-9fe6e1ceff9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:23.478490 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-0cbaa757-16dc-410c-8439-c9644317581c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fb3ad1adbbb4c8985a71b79b080b753'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:23.534243 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdf1ea-da8b-42e5-8928-9fe6e1ceff9f None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:23.543403 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec2ad6-1bc0-48dd-9400-71af0941774e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:23.547704 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec2ad6-1bc0-48dd-9400-71af0941774e admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:23.547704 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec2ad6-1bc0-48dd-9400-71af0941774e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:23.547704 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec2ad6-1bc0-48dd-9400-71af0941774e admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:23.547704 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec2ad6-1bc0-48dd-9400-71af0941774e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:23.548128 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec2ad6-1bc0-48dd-9400-71af0941774e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:23.569069 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-0cbaa757-16dc-410c-8439-c9644317581c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fb3ad1adbbb4c8985a71b79b080b753'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:23.569781 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-0cbaa757-16dc-410c-8439-c9644317581c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fb3ad1adbbb4c8985a71b79b080b753'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:23.598426 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-0cbaa757-16dc-410c-8439-c9644317581c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fb3ad1adbbb4c8985a71b79b080b753'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:23.599589 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-0cbaa757-16dc-410c-8439-c9644317581c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fb3ad1adbbb4c8985a71b79b080b753'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:23.613668 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 914/1819] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:13:23 2022] DELETE /identity/v3/users/9fb3ad1adbbb4c8985a71b79b080b753 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:13:23.618694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0045d30-56f0-42b3-8475-105fc0f397ca None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:23.619018 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0045d30-56f0-42b3-8475-105fc0f397ca None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:23.619305 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0045d30-56f0-42b3-8475-105fc0f397ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:23.646868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-e0045d30-56f0-42b3-8475-105fc0f397ca None None] MFA Rules not processed for user `a3fa4a6c16274c18849869b55b33ac62`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:23.648615 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-e0045d30-56f0-42b3-8475-105fc0f397ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:23.694131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 906/1820] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:13:23 2022] POST /identity/v3/projects => generated 387 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:23.695147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec2ad6-1bc0-48dd-9400-71af0941774e admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:23.702134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0fec2ad6-1bc0-48dd-9400-71af0941774e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:23.838373 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec2ad6-1bc0-48dd-9400-71af0941774e admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:23.850116 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 915/1821] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Aug 3 05:13:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:23.850116 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0045d30-56f0-42b3-8475-105fc0f397ca None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:23.850333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e898a8c1-0087-4a6c-b056-6c9ece9d13f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:23.850421 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-e0045d30-56f0-42b3-8475-105fc0f397ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:23.850490 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898a8c1-0087-4a6c-b056-6c9ece9d13f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:23.850490 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898a8c1-0087-4a6c-b056-6c9ece9d13f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:23.850490 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898a8c1-0087-4a6c-b056-6c9ece9d13f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:23.850490 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e898a8c1-0087-4a6c-b056-6c9ece9d13f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:23.850490 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e898a8c1-0087-4a6c-b056-6c9ece9d13f4 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:23.930654 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0045d30-56f0-42b3-8475-105fc0f397ca None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:23.936964 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd66da-9389-4051-b324-cef3d305e251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:23.937705 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd66da-9389-4051-b324-cef3d305e251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:23.938056 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd66da-9389-4051-b324-cef3d305e251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:23.938573 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd66da-9389-4051-b324-cef3d305e251 admin admin] PATH_INFO: `/v3/users/17707aa716d5442797536630b6e73112` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:23.940417 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd66da-9389-4051-b324-cef3d305e251 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17707aa716d5442797536630b6e73112)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:23.940999 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 907/1822] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:13:23 2022] POST /identity/v3/users => generated 300 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:23.945884 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd66da-9389-4051-b324-cef3d305e251 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:23.955544 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e898a8c1-0087-4a6c-b056-6c9ece9d13f4 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:23.970510 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e898a8c1-0087-4a6c-b056-6c9ece9d13f4 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:23.980565 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba223d3c-0d19-4f6c-b515-6cc4d8311804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:23.981830 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba223d3c-0d19-4f6c-b515-6cc4d8311804 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:23.982155 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba223d3c-0d19-4f6c-b515-6cc4d8311804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:23.982637 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba223d3c-0d19-4f6c-b515-6cc4d8311804 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:23.983336 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba223d3c-0d19-4f6c-b515-6cc4d8311804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:23.985051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba223d3c-0d19-4f6c-b515-6cc4d8311804 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:24.007019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 908/1823] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:24.014199 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba223d3c-0d19-4f6c-b515-6cc4d8311804 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:24.018567 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ba223d3c-0d19-4f6c-b515-6cc4d8311804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:24.106739 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba223d3c-0d19-4f6c-b515-6cc4d8311804 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:24.112482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace14a3-8561-405f-bd5c-0ac437687be5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:24.113145 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace14a3-8561-405f-bd5c-0ac437687be5 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:24.113432 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace14a3-8561-405f-bd5c-0ac437687be5 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:24.113741 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace14a3-8561-405f-bd5c-0ac437687be5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:24.118886 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:24.123248 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace14a3-8561-405f-bd5c-0ac437687be5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:24.124669 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace14a3-8561-405f-bd5c-0ac437687be5 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:24.135219 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 909/1824] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:13:24.135987 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace14a3-8561-405f-bd5c-0ac437687be5 service nova] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:24.146679 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace14a3-8561-405f-bd5c-0ac437687be5 service nova] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:24.151276 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d463b-f56b-4cc4-a54f-0fb45f2cd8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:24.152294 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d463b-f56b-4cc4-a54f-0fb45f2cd8e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:24.152587 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d463b-f56b-4cc4-a54f-0fb45f2cd8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:24.153278 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d463b-f56b-4cc4-a54f-0fb45f2cd8e0 admin admin] PATH_INFO: `/v3/projects/8be32a432f40442689dcb32fc0450f66/users/1fd1cf3ffdb34f38a004145ca5fff164/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:24.170363 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d463b-f56b-4cc4-a54f-0fb45f2cd8e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8be32a432f40442689dcb32fc0450f66, user_id=1fd1cf3ffdb34f38a004145ca5fff164, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:24.173481 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d463b-f56b-4cc4-a54f-0fb45f2cd8e0 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:24.222971 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-65cd66da-9389-4051-b324-cef3d305e251 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17707aa716d5442797536630b6e73112'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:24.319761 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 910/1825] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:24 2022] PUT /identity/v3/projects/8be32a432f40442689dcb32fc0450f66/users/1fd1cf3ffdb34f38a004145ca5fff164/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:24.323170 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d463b-f56b-4cc4-a54f-0fb45f2cd8e0 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:24.323750 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-144d463b-f56b-4cc4-a54f-0fb45f2cd8e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:24.336120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-65cd66da-9389-4051-b324-cef3d305e251 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17707aa716d5442797536630b6e73112'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:24.336887 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-65cd66da-9389-4051-b324-cef3d305e251 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17707aa716d5442797536630b6e73112'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:24.351735 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-65cd66da-9389-4051-b324-cef3d305e251 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17707aa716d5442797536630b6e73112'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:24.352513 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-65cd66da-9389-4051-b324-cef3d305e251 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17707aa716d5442797536630b6e73112'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:24.365362 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 916/1826] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:13:23 2022] DELETE /identity/v3/users/17707aa716d5442797536630b6e73112 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:24.366456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd66da-9389-4051-b324-cef3d305e251 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:24.367646 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-65cd66da-9389-4051-b324-cef3d305e251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:24.392164 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d463b-f56b-4cc4-a54f-0fb45f2cd8e0 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:24.432832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd66da-9389-4051-b324-cef3d305e251 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:24.437910 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873e769-ca6e-4f96-b468-94957e421249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:24.438540 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873e769-ca6e-4f96-b468-94957e421249 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:24.438824 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873e769-ca6e-4f96-b468-94957e421249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:24.443732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873e769-ca6e-4f96-b468-94957e421249 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:24.443732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b873e769-ca6e-4f96-b468-94957e421249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:24.443732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b873e769-ca6e-4f96-b468-94957e421249 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:24.450989 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3e367-5310-4fc4-bd19-1e3e627f57d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:24.451718 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3e367-5310-4fc4-bd19-1e3e627f57d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:24.452327 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3e367-5310-4fc4-bd19-1e3e627f57d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:24.452437 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 917/1827] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:24.452579 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3e367-5310-4fc4-bd19-1e3e627f57d6 admin admin] PATH_INFO: `/v3/projects/f860d7f3a8504bd7b2c36068a020489b` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:24.453375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608eae5-da12-4524-b6e1-45fae676724d None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:24.453673 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608eae5-da12-4524-b6e1-45fae676724d None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:24.453956 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608eae5-da12-4524-b6e1-45fae676724d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:24.454057 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d3e367-5310-4fc4-bd19-1e3e627f57d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f860d7f3a8504bd7b2c36068a020489b)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:24.456290 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d3e367-5310-4fc4-bd19-1e3e627f57d6 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:24.473030 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-5608eae5-da12-4524-b6e1-45fae676724d None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:24.474515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-5608eae5-da12-4524-b6e1-45fae676724d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:24.482461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 918/1828] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:24.483266 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5608eae5-da12-4524-b6e1-45fae676724d None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:24.492677 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5608eae5-da12-4524-b6e1-45fae676724d None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:24.497441 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41a7d0-8414-409c-a928-0c30f0d9a946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:24.498121 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41a7d0-8414-409c-a928-0c30f0d9a946 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:24.498403 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41a7d0-8414-409c-a928-0c30f0d9a946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:24.498681 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41a7d0-8414-409c-a928-0c30f0d9a946 admin admin] PATH_INFO: `/v3/projects/8be32a432f40442689dcb32fc0450f66/users/1fd1cf3ffdb34f38a004145ca5fff164/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:24.500682 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41a7d0-8414-409c-a928-0c30f0d9a946 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8be32a432f40442689dcb32fc0450f66, user_id=1fd1cf3ffdb34f38a004145ca5fff164, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:24.503413 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41a7d0-8414-409c-a928-0c30f0d9a946 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:24.514267 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 919/1829] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:24 2022] PUT /identity/v3/projects/8be32a432f40442689dcb32fc0450f66/users/1fd1cf3ffdb34f38a004145ca5fff164/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:24.515002 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41a7d0-8414-409c-a928-0c30f0d9a946 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:24.516777 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-7d41a7d0-8414-409c-a928-0c30f0d9a946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:24.578374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41a7d0-8414-409c-a928-0c30f0d9a946 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:24.582695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e7b649-04e0-4afc-b4a5-0d3bad60ea69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:24.583367 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7b649-04e0-4afc-b4a5-0d3bad60ea69 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:24.583664 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7b649-04e0-4afc-b4a5-0d3bad60ea69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:24.583941 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7b649-04e0-4afc-b4a5-0d3bad60ea69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:24.607453 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c6d3e367-5310-4fc4-bd19-1e3e627f57d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f860d7f3a8504bd7b2c36068a020489b', 'request_id': 'req-c6d3e367-5310-4fc4-bd19-1e3e627f57d6'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:24.654956 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e7b649-04e0-4afc-b4a5-0d3bad60ea69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:24.656424 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e7b649-04e0-4afc-b4a5-0d3bad60ea69 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:24.670294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 920/1830] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:24.671577 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c113f9-7bf1-4fc5-8c61-c2993b77b6d7 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:24.677881 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c113f9-7bf1-4fc5-8c61-c2993b77b6d7 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:24.677881 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c113f9-7bf1-4fc5-8c61-c2993b77b6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:24.678087 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c6d3e367-5310-4fc4-bd19-1e3e627f57d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f860d7f3a8504bd7b2c36068a020489b', 'request_id': 'req-c6d3e367-5310-4fc4-bd19-1e3e627f57d6'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:24.680736 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c6d3e367-5310-4fc4-bd19-1e3e627f57d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f860d7f3a8504bd7b2c36068a020489b', 'request_id': 'req-c6d3e367-5310-4fc4-bd19-1e3e627f57d6'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:24.709256 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c6d3e367-5310-4fc4-bd19-1e3e627f57d6 admin admin] The token cache is being invalidate because project f860d7f3a8504bd7b2c36068a020489b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:13:24.709800 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c6d3e367-5310-4fc4-bd19-1e3e627f57d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:24.714085 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c6d3e367-5310-4fc4-bd19-1e3e627f57d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:24.721121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 911/1831] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:13:24 2022] DELETE /identity/v3/projects/f860d7f3a8504bd7b2c36068a020489b => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:24.722683 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9641b4-2436-42d8-a9c1-5cbdc6ab48d4 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:24.723006 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9641b4-2436-42d8-a9c1-5cbdc6ab48d4 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:24.723354 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9641b4-2436-42d8-a9c1-5cbdc6ab48d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:24.743322 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-14c113f9-7bf1-4fc5-8c61-c2993b77b6d7 None None] MFA Rules not processed for user `1fd1cf3ffdb34f38a004145ca5fff164`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:24.745064 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-14c113f9-7bf1-4fc5-8c61-c2993b77b6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:24.761950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:24.763283 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-5a9641b4-2436-42d8-a9c1-5cbdc6ab48d4 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:24.765248 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-5a9641b4-2436-42d8-a9c1-5cbdc6ab48d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:24.864923 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 921/1832] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:24 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:24.873017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 912/1833] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:24.877052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c113f9-7bf1-4fc5-8c61-c2993b77b6d7 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:24.879296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-14c113f9-7bf1-4fc5-8c61-c2993b77b6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:24.881741 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a221e4b-0870-4232-b4b1-75d40c7e3628 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:24.882148 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a221e4b-0870-4232-b4b1-75d40c7e3628 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:24.882524 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a221e4b-0870-4232-b4b1-75d40c7e3628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:24.904122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-2a221e4b-0870-4232-b4b1-75d40c7e3628 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:24.905131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2a221e4b-0870-4232-b4b1-75d40c7e3628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:24.924905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 913/1834] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:24.929945 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a221e4b-0870-4232-b4b1-75d40c7e3628 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:24.955096 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a221e4b-0870-4232-b4b1-75d40c7e3628 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:24.962689 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f69c6a-eddd-4b1d-980f-3df32042ad76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:24.967353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f69c6a-eddd-4b1d-980f-3df32042ad76 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:24.967353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f69c6a-eddd-4b1d-980f-3df32042ad76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:24.967353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f69c6a-eddd-4b1d-980f-3df32042ad76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:24.967353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f69c6a-eddd-4b1d-980f-3df32042ad76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:13:24.967353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f69c6a-eddd-4b1d-980f-3df32042ad76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:24.968753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f69c6a-eddd-4b1d-980f-3df32042ad76 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:24.973517 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c113f9-7bf1-4fc5-8c61-c2993b77b6d7 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:24.977848 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5bf90-f493-4c8c-80ec-35685749ef2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:24.978555 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5bf90-f493-4c8c-80ec-35685749ef2c service cinder] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:24.979011 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5bf90-f493-4c8c-80ec-35685749ef2c service cinder] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:24.979357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5bf90-f493-4c8c-80ec-35685749ef2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:24.984034 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 914/1835] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:13:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:13:24.989507 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f69c6a-eddd-4b1d-980f-3df32042ad76 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:25.003212 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f69c6a-eddd-4b1d-980f-3df32042ad76 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:25.009423 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c5bf90-f493-4c8c-80ec-35685749ef2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:25.012031 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c5bf90-f493-4c8c-80ec-35685749ef2c service cinder] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:25.012436 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae0c1c-295b-40e6-aa3d-df65ff36a51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:25.013492 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae0c1c-295b-40e6-aa3d-df65ff36a51f admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:25.013774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae0c1c-295b-40e6-aa3d-df65ff36a51f admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:25.014080 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae0c1c-295b-40e6-aa3d-df65ff36a51f admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:25.016350 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ae0c1c-295b-40e6-aa3d-df65ff36a51f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:25.016814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ae0c1c-295b-40e6-aa3d-df65ff36a51f admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:25.026537 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 922/1836] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:24 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:25.143429 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 915/1837] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:13:24 2022] POST /identity/v3/projects => generated 367 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:25.145614 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5bf90-f493-4c8c-80ec-35685749ef2c service cinder] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:25.157765 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5bf90-f493-4c8c-80ec-35685749ef2c service cinder] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:25.176533 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae0c1c-295b-40e6-aa3d-df65ff36a51f admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:25.178561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-44ae0c1c-295b-40e6-aa3d-df65ff36a51f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:25.245214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f59669-083d-4c8f-9cba-e0c294733e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:25.246060 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f59669-083d-4c8f-9cba-e0c294733e0c admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:25.246443 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f59669-083d-4c8f-9cba-e0c294733e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:25.246876 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f59669-083d-4c8f-9cba-e0c294733e0c admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:25.247646 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f59669-083d-4c8f-9cba-e0c294733e0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:25.249941 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f59669-083d-4c8f-9cba-e0c294733e0c admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:25.295695 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae0c1c-295b-40e6-aa3d-df65ff36a51f admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:25.302898 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd1a54-e306-4e56-b297-17b2ce28c07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:25.310286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd1a54-e306-4e56-b297-17b2ce28c07a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:25.310286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd1a54-e306-4e56-b297-17b2ce28c07a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:25.310286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd1a54-e306-4e56-b297-17b2ce28c07a admin admin] PATH_INFO: `/v3/users/807ae4811a134a5f979cda98adaf7ff3` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:25.310286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfd1a54-e306-4e56-b297-17b2ce28c07a admin admin] RBAC: Authorizing `identity:delete_user(user_id=807ae4811a134a5f979cda98adaf7ff3)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:25.310286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfd1a54-e306-4e56-b297-17b2ce28c07a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:25.348980 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 923/1838] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:13:25 2022] POST /identity/v3/users => generated 284 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:25.350185 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f59669-083d-4c8f-9cba-e0c294733e0c admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:25.352910 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-e7f59669-083d-4c8f-9cba-e0c294733e0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:25.370260 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f59669-083d-4c8f-9cba-e0c294733e0c admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:25.376373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab46c6-dbc5-4681-8476-c4b3f656db1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:25.377121 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab46c6-dbc5-4681-8476-c4b3f656db1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:25.377472 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab46c6-dbc5-4681-8476-c4b3f656db1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:25.377820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab46c6-dbc5-4681-8476-c4b3f656db1c admin admin] PATH_INFO: `/v3/projects/fd87dd9e00554bc68e512faeb5ee0de4` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:25.380579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab46c6-dbc5-4681-8476-c4b3f656db1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd87dd9e00554bc68e512faeb5ee0de4)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:25.382878 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab46c6-dbc5-4681-8476-c4b3f656db1c admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:25.516947 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-6dab46c6-dbc5-4681-8476-c4b3f656db1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd87dd9e00554bc68e512faeb5ee0de4', 'request_id': 'req-6dab46c6-dbc5-4681-8476-c4b3f656db1c'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:25.528458 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-6dab46c6-dbc5-4681-8476-c4b3f656db1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd87dd9e00554bc68e512faeb5ee0de4', 'request_id': 'req-6dab46c6-dbc5-4681-8476-c4b3f656db1c'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:25.529380 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7cfd1a54-e306-4e56-b297-17b2ce28c07a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '807ae4811a134a5f979cda98adaf7ff3'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:25.655120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-6dab46c6-dbc5-4681-8476-c4b3f656db1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd87dd9e00554bc68e512faeb5ee0de4', 'request_id': 'req-6dab46c6-dbc5-4681-8476-c4b3f656db1c'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:25.669660 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-6dab46c6-dbc5-4681-8476-c4b3f656db1c admin admin] The token cache is being invalidate because project fd87dd9e00554bc68e512faeb5ee0de4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:13:25.670047 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-6dab46c6-dbc5-4681-8476-c4b3f656db1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:25.670795 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-6dab46c6-dbc5-4681-8476-c4b3f656db1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:25.675166 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 924/1839] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:13:25 2022] DELETE /identity/v3/projects/fd87dd9e00554bc68e512faeb5ee0de4 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:25.676326 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab46c6-dbc5-4681-8476-c4b3f656db1c admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:25.678252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-6dab46c6-dbc5-4681-8476-c4b3f656db1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:25.718907 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7cfd1a54-e306-4e56-b297-17b2ce28c07a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '807ae4811a134a5f979cda98adaf7ff3'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:25.720096 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7cfd1a54-e306-4e56-b297-17b2ce28c07a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '807ae4811a134a5f979cda98adaf7ff3'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:25.739119 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7cfd1a54-e306-4e56-b297-17b2ce28c07a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '807ae4811a134a5f979cda98adaf7ff3'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:25.739886 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7cfd1a54-e306-4e56-b297-17b2ce28c07a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '807ae4811a134a5f979cda98adaf7ff3'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:25.758737 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 916/1840] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:13:25 2022] DELETE /identity/v3/users/807ae4811a134a5f979cda98adaf7ff3 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:13:25.759070 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab46c6-dbc5-4681-8476-c4b3f656db1c admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:25.760048 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd1a54-e306-4e56-b297-17b2ce28c07a admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:25.772391 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7cfd1a54-e306-4e56-b297-17b2ce28c07a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:25.854541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c3df4e-9c9f-4064-8d3f-7d98fe696bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:25.866057 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3df4e-9c9f-4064-8d3f-7d98fe696bee admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:25.866057 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3df4e-9c9f-4064-8d3f-7d98fe696bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:25.866057 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3df4e-9c9f-4064-8d3f-7d98fe696bee admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:25.866057 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c3df4e-9c9f-4064-8d3f-7d98fe696bee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:25.866057 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c3df4e-9c9f-4064-8d3f-7d98fe696bee admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:25.879617 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 925/1841] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:25.879617 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19525f-80cc-4e9f-9a80-3bb877957d91 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:25.879617 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19525f-80cc-4e9f-9a80-3bb877957d91 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:25.879617 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19525f-80cc-4e9f-9a80-3bb877957d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:25.909109 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd1a54-e306-4e56-b297-17b2ce28c07a admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:25.919274 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-0c19525f-80cc-4e9f-9a80-3bb877957d91 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:25.923144 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-0c19525f-80cc-4e9f-9a80-3bb877957d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:25.934620 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 926/1842] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:25.936211 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19525f-80cc-4e9f-9a80-3bb877957d91 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:25.937928 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3449e8b-6a07-401f-aa24-d2cc01446aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:25.939354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-0c19525f-80cc-4e9f-9a80-3bb877957d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:25.941969 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3449e8b-6a07-401f-aa24-d2cc01446aac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:25.942309 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3449e8b-6a07-401f-aa24-d2cc01446aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:25.942595 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3449e8b-6a07-401f-aa24-d2cc01446aac admin admin] PATH_INFO: `/v3/users/ee80bae4563543e58872ce0ed1f08044` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:25.950060 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3449e8b-6a07-401f-aa24-d2cc01446aac admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee80bae4563543e58872ce0ed1f08044)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:25.952837 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3449e8b-6a07-401f-aa24-d2cc01446aac admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:25.977735 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19525f-80cc-4e9f-9a80-3bb877957d91 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:25.985801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9377c-ce2f-4982-b432-b726ea4f2c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:25.987138 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9377c-ce2f-4982-b432-b726ea4f2c2d admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:25.987423 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9377c-ce2f-4982-b432-b726ea4f2c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:25.987722 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9377c-ce2f-4982-b432-b726ea4f2c2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:25.989009 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9377c-ce2f-4982-b432-b726ea4f2c2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:25.990965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9377c-ce2f-4982-b432-b726ea4f2c2d admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:26.006740 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 927/1843] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:26.006740 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9377c-ce2f-4982-b432-b726ea4f2c2d admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:26.009522 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-cdf9377c-ce2f-4982-b432-b726ea4f2c2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:26.063118 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:26.128722 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9377c-ce2f-4982-b432-b726ea4f2c2d admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:26.128722 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b038b7e-70c9-4924-9c0b-63d86d869998 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:26.128884 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b038b7e-70c9-4924-9c0b-63d86d869998 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:26.129038 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b038b7e-70c9-4924-9c0b-63d86d869998 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:26.129313 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b038b7e-70c9-4924-9c0b-63d86d869998 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:26.145261 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b038b7e-70c9-4924-9c0b-63d86d869998 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:26.147289 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b038b7e-70c9-4924-9c0b-63d86d869998 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:26.166144 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 928/1844] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:26.167578 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b038b7e-70c9-4924-9c0b-63d86d869998 service neutron] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:26.179376 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b038b7e-70c9-4924-9c0b-63d86d869998 service neutron] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:26.185130 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71463b8b-632d-4650-9145-b275ff138bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:26.185881 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71463b8b-632d-4650-9145-b275ff138bb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:26.186169 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71463b8b-632d-4650-9145-b275ff138bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:26.186497 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71463b8b-632d-4650-9145-b275ff138bb5 admin admin] PATH_INFO: `/v3/projects/2fd8d36a8f474a2ea2fd04edf013b2a8/users/16a650a612ea4d3495e28ba33686b709/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:26.208556 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d3449e8b-6a07-401f-aa24-d2cc01446aac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee80bae4563543e58872ce0ed1f08044'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:26.219579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71463b8b-632d-4650-9145-b275ff138bb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fd8d36a8f474a2ea2fd04edf013b2a8, user_id=16a650a612ea4d3495e28ba33686b709, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:26.223750 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71463b8b-632d-4650-9145-b275ff138bb5 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:26.273443 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d3449e8b-6a07-401f-aa24-d2cc01446aac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee80bae4563543e58872ce0ed1f08044'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:26.274567 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d3449e8b-6a07-401f-aa24-d2cc01446aac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee80bae4563543e58872ce0ed1f08044'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:26.293655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d3449e8b-6a07-401f-aa24-d2cc01446aac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee80bae4563543e58872ce0ed1f08044'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:26.295657 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d3449e8b-6a07-401f-aa24-d2cc01446aac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee80bae4563543e58872ce0ed1f08044'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:26.322341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 917/1846] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:13:25 2022] DELETE /identity/v3/users/ee80bae4563543e58872ce0ed1f08044 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:26.324413 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46110115-7400-4c69-aaaa-1c8a1656bdba None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:26.324715 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46110115-7400-4c69-aaaa-1c8a1656bdba None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:26.324920 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 929/1846] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:26 2022] PUT /identity/v3/projects/2fd8d36a8f474a2ea2fd04edf013b2a8/users/16a650a612ea4d3495e28ba33686b709/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:26.325077 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46110115-7400-4c69-aaaa-1c8a1656bdba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:26.362406 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-46110115-7400-4c69-aaaa-1c8a1656bdba None None] MFA Rules not processed for user `16a650a612ea4d3495e28ba33686b709`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:26.363856 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-46110115-7400-4c69-aaaa-1c8a1656bdba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:26.444159 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 918/1847] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:26.448739 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71463b8b-632d-4650-9145-b275ff138bb5 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:26.451047 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-71463b8b-632d-4650-9145-b275ff138bb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:26.515362 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:26.519817 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71463b8b-632d-4650-9145-b275ff138bb5 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:26.579071 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06010b4c-82d9-4d67-915e-fd4c74bcd3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:26.579635 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06010b4c-82d9-4d67-915e-fd4c74bcd3fc admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:26.579918 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06010b4c-82d9-4d67-915e-fd4c74bcd3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:26.580200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06010b4c-82d9-4d67-915e-fd4c74bcd3fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:26.580863 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06010b4c-82d9-4d67-915e-fd4c74bcd3fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:26.584000 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06010b4c-82d9-4d67-915e-fd4c74bcd3fc admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:26.639767 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:26.714043 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 930/1848] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:13:26 2022] POST /identity/v3/projects => generated 367 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:13:26.716899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46110115-7400-4c69-aaaa-1c8a1656bdba None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:26.735931 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46110115-7400-4c69-aaaa-1c8a1656bdba None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:26.833422 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45416ae7-e59f-4db2-abd6-93926b36e991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:26.833422 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45416ae7-e59f-4db2-abd6-93926b36e991 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:26.833422 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45416ae7-e59f-4db2-abd6-93926b36e991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:26.833422 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45416ae7-e59f-4db2-abd6-93926b36e991 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:26.834045 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45416ae7-e59f-4db2-abd6-93926b36e991 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:26.838909 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45416ae7-e59f-4db2-abd6-93926b36e991 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:26.950896 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 919/1849] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:13:26 2022] POST /identity/v3/users => generated 290 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:26.953254 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06010b4c-82d9-4d67-915e-fd4c74bcd3fc admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:26.964569 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06010b4c-82d9-4d67-915e-fd4c74bcd3fc admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:26.972821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07948467-7a77-406c-b315-bc37f40574b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:26.974120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07948467-7a77-406c-b315-bc37f40574b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:26.974533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07948467-7a77-406c-b315-bc37f40574b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:26.974820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07948467-7a77-406c-b315-bc37f40574b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:26.983119 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07948467-7a77-406c-b315-bc37f40574b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:26.983119 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07948467-7a77-406c-b315-bc37f40574b2 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:26.984231 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45416ae7-e59f-4db2-abd6-93926b36e991 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:26.985957 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-45416ae7-e59f-4db2-abd6-93926b36e991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:27.002618 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45416ae7-e59f-4db2-abd6-93926b36e991 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:27.013890 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 931/1850] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:13:27.014871 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07948467-7a77-406c-b315-bc37f40574b2 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:27.022429 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c13849-bd2f-4940-95ee-50c62ab40fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:27.022429 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c13849-bd2f-4940-95ee-50c62ab40fad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:27.022429 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c13849-bd2f-4940-95ee-50c62ab40fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:27.022429 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c13849-bd2f-4940-95ee-50c62ab40fad admin admin] PATH_INFO: `/v3/projects/7e3d7b457546483381a91a3d9a55a95f` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:27.022889 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c13849-bd2f-4940-95ee-50c62ab40fad admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e3d7b457546483381a91a3d9a55a95f)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:27.022889 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c13849-bd2f-4940-95ee-50c62ab40fad admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:27.055740 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07948467-7a77-406c-b315-bc37f40574b2 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:27.057352 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b445d-a15e-4c28-906d-8b0e7c2d0990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:27.058144 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b445d-a15e-4c28-906d-8b0e7c2d0990 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:27.058515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b445d-a15e-4c28-906d-8b0e7c2d0990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:27.058882 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b445d-a15e-4c28-906d-8b0e7c2d0990 admin admin] PATH_INFO: `/v3/projects/264942a659dc488fa1bc2cf11fc1eda9/users/5d2be48511224fc5aa1dd7e92c54b7e9/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:27.090174 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95b445d-a15e-4c28-906d-8b0e7c2d0990 admin admin] RBAC: Authorizing `identity:create_grant(project_id=264942a659dc488fa1bc2cf11fc1eda9, user_id=5d2be48511224fc5aa1dd7e92c54b7e9, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:27.090174 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95b445d-a15e-4c28-906d-8b0e7c2d0990 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:27.210258 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 932/1851] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:27 2022] PUT /identity/v3/projects/264942a659dc488fa1bc2cf11fc1eda9/users/5d2be48511224fc5aa1dd7e92c54b7e9/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:27.210868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b445d-a15e-4c28-906d-8b0e7c2d0990 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:27.212597 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-c95b445d-a15e-4c28-906d-8b0e7c2d0990 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:27.227238 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b3c13849-bd2f-4940-95ee-50c62ab40fad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e3d7b457546483381a91a3d9a55a95f', 'request_id': 'req-b3c13849-bd2f-4940-95ee-50c62ab40fad'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:27.285527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b445d-a15e-4c28-906d-8b0e7c2d0990 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:27.312178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b3c13849-bd2f-4940-95ee-50c62ab40fad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e3d7b457546483381a91a3d9a55a95f', 'request_id': 'req-b3c13849-bd2f-4940-95ee-50c62ab40fad'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:27.319616 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b3c13849-bd2f-4940-95ee-50c62ab40fad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e3d7b457546483381a91a3d9a55a95f', 'request_id': 'req-b3c13849-bd2f-4940-95ee-50c62ab40fad'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:27.334421 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b3c13849-bd2f-4940-95ee-50c62ab40fad admin admin] The token cache is being invalidate because project 7e3d7b457546483381a91a3d9a55a95f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:13:27.334719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b3c13849-bd2f-4940-95ee-50c62ab40fad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:27.335434 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b3c13849-bd2f-4940-95ee-50c62ab40fad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:27.337146 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 920/1852] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:13:26 2022] DELETE /identity/v3/projects/7e3d7b457546483381a91a3d9a55a95f => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:13:27.337941 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c13849-bd2f-4940-95ee-50c62ab40fad admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:27.340995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b3c13849-bd2f-4940-95ee-50c62ab40fad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:27.367004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308efa5d-e404-4951-af09-dc77fdcab289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:27.367730 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308efa5d-e404-4951-af09-dc77fdcab289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:27.368016 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308efa5d-e404-4951-af09-dc77fdcab289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:27.368299 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308efa5d-e404-4951-af09-dc77fdcab289 admin admin] PATH_INFO: `/v3/projects/fbc9dc1e38df41a39bac79e2ce3d212e` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:27.369567 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308efa5d-e404-4951-af09-dc77fdcab289 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbc9dc1e38df41a39bac79e2ce3d212e)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:27.380139 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308efa5d-e404-4951-af09-dc77fdcab289 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:27.437776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c13849-bd2f-4940-95ee-50c62ab40fad admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:27.442544 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32fc9f-1662-40f0-b217-c067a252d17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:27.443205 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32fc9f-1662-40f0-b217-c067a252d17d admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:27.443495 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32fc9f-1662-40f0-b217-c067a252d17d admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:27.443774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32fc9f-1662-40f0-b217-c067a252d17d admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:27.444475 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b32fc9f-1662-40f0-b217-c067a252d17d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:27.446089 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b32fc9f-1662-40f0-b217-c067a252d17d admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:27.456811 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 921/1853] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:27.457552 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32fc9f-1662-40f0-b217-c067a252d17d admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:27.463430 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9b32fc9f-1662-40f0-b217-c067a252d17d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:27.547516 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32fc9f-1662-40f0-b217-c067a252d17d admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:27.547516 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c5738-4fef-4d05-bbd3-237a9421142c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:27.547516 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c5738-4fef-4d05-bbd3-237a9421142c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:27.547516 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c5738-4fef-4d05-bbd3-237a9421142c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:27.547881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c5738-4fef-4d05-bbd3-237a9421142c admin admin] PATH_INFO: `/v3/users/1fd1cf3ffdb34f38a004145ca5fff164` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:27.547881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c5738-4fef-4d05-bbd3-237a9421142c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fd1cf3ffdb34f38a004145ca5fff164)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:27.547881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c5738-4fef-4d05-bbd3-237a9421142c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:28.111081 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-308efa5d-e404-4951-af09-dc77fdcab289 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbc9dc1e38df41a39bac79e2ce3d212e', 'request_id': 'req-308efa5d-e404-4951-af09-dc77fdcab289'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:28.117895 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-308efa5d-e404-4951-af09-dc77fdcab289 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbc9dc1e38df41a39bac79e2ce3d212e', 'request_id': 'req-308efa5d-e404-4951-af09-dc77fdcab289'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:28.519743 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-308efa5d-e404-4951-af09-dc77fdcab289 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbc9dc1e38df41a39bac79e2ce3d212e', 'request_id': 'req-308efa5d-e404-4951-af09-dc77fdcab289'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:28.532439 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-308efa5d-e404-4951-af09-dc77fdcab289 admin admin] The token cache is being invalidate because project fbc9dc1e38df41a39bac79e2ce3d212e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:13:28.532994 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-308efa5d-e404-4951-af09-dc77fdcab289 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:28.533757 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-308efa5d-e404-4951-af09-dc77fdcab289 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:28.535899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-553c5738-4fef-4d05-bbd3-237a9421142c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fd1cf3ffdb34f38a004145ca5fff164'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:28.536355 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 933/1854] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:13:27 2022] DELETE /identity/v3/projects/fbc9dc1e38df41a39bac79e2ce3d212e => generated 0 bytes in 1325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:28.536922 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308efa5d-e404-4951-af09-dc77fdcab289 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:28.540840 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-308efa5d-e404-4951-af09-dc77fdcab289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:28.610145 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308efa5d-e404-4951-af09-dc77fdcab289 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:28.615423 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16339fe1-a13f-4ccb-9b29-019f4a71d7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:28.616091 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16339fe1-a13f-4ccb-9b29-019f4a71d7ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:28.616370 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16339fe1-a13f-4ccb-9b29-019f4a71d7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:28.616643 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16339fe1-a13f-4ccb-9b29-019f4a71d7ba admin admin] PATH_INFO: `/v3/projects/264942a659dc488fa1bc2cf11fc1eda9/users/5d2be48511224fc5aa1dd7e92c54b7e9/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:28.618663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16339fe1-a13f-4ccb-9b29-019f4a71d7ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=264942a659dc488fa1bc2cf11fc1eda9, user_id=5d2be48511224fc5aa1dd7e92c54b7e9, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:28.621351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16339fe1-a13f-4ccb-9b29-019f4a71d7ba admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:28.627262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-553c5738-4fef-4d05-bbd3-237a9421142c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fd1cf3ffdb34f38a004145ca5fff164'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:28.628110 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-553c5738-4fef-4d05-bbd3-237a9421142c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fd1cf3ffdb34f38a004145ca5fff164'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:28.633699 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 934/1855] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:28 2022] PUT /identity/v3/projects/264942a659dc488fa1bc2cf11fc1eda9/users/5d2be48511224fc5aa1dd7e92c54b7e9/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:28.635216 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ade1ea-fa08-4dd2-bdda-abf51327364a None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:28.635615 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ade1ea-fa08-4dd2-bdda-abf51327364a None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:28.635894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ade1ea-fa08-4dd2-bdda-abf51327364a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:28.643345 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-553c5738-4fef-4d05-bbd3-237a9421142c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fd1cf3ffdb34f38a004145ca5fff164'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:28.644629 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-553c5738-4fef-4d05-bbd3-237a9421142c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fd1cf3ffdb34f38a004145ca5fff164'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:28.657083 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 922/1856] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:13:27 2022] DELETE /identity/v3/users/1fd1cf3ffdb34f38a004145ca5fff164 => generated 0 bytes in 1200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:28.658414 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2438afc-eb4c-4ee1-a654-fb2fe1ce5beb None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:28.658708 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2438afc-eb4c-4ee1-a654-fb2fe1ce5beb None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:28.659025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2438afc-eb4c-4ee1-a654-fb2fe1ce5beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:28.662723 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-50ade1ea-fa08-4dd2-bdda-abf51327364a None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:28.667723 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-50ade1ea-fa08-4dd2-bdda-abf51327364a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:28.693927 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-c2438afc-eb4c-4ee1-a654-fb2fe1ce5beb None None] MFA Rules not processed for user `5d2be48511224fc5aa1dd7e92c54b7e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:28.695219 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c2438afc-eb4c-4ee1-a654-fb2fe1ce5beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:28.729957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 935/1857] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:28.731819 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:28.737410 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529bb4c-f106-45a0-90b3-282202d8dde5 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:28.738454 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529bb4c-f106-45a0-90b3-282202d8dde5 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:28.738989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529bb4c-f106-45a0-90b3-282202d8dde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:28.768126 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-b529bb4c-f106-45a0-90b3-282202d8dde5 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:28.769409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-b529bb4c-f106-45a0-90b3-282202d8dde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:28.777737 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 936/1858] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:13:28.787744 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529bb4c-f106-45a0-90b3-282202d8dde5 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:28.789621 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 923/1859] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:28 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:28.811156 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2438afc-eb4c-4ee1-a654-fb2fe1ce5beb None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:28.814053 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529bb4c-f106-45a0-90b3-282202d8dde5 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:28.820267 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c2438afc-eb4c-4ee1-a654-fb2fe1ce5beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:28.829017 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008ab89-7ee0-4378-aac0-32fec7eb588c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:28.830593 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008ab89-7ee0-4378-aac0-32fec7eb588c admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:28.832392 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008ab89-7ee0-4378-aac0-32fec7eb588c admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:28.832967 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008ab89-7ee0-4378-aac0-32fec7eb588c admin admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:28.834569 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008ab89-7ee0-4378-aac0-32fec7eb588c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:13:28.835109 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008ab89-7ee0-4378-aac0-32fec7eb588c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:28.837967 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008ab89-7ee0-4378-aac0-32fec7eb588c admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:28.860095 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 937/1860] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:13:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 05:13:28.863392 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008ab89-7ee0-4378-aac0-32fec7eb588c admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:28.889741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008ab89-7ee0-4378-aac0-32fec7eb588c admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:28.906076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf039b-5a5d-4921-a597-bafc9323ec14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:28.906721 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf039b-5a5d-4921-a597-bafc9323ec14 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:28.907145 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf039b-5a5d-4921-a597-bafc9323ec14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:28.907497 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf039b-5a5d-4921-a597-bafc9323ec14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:28.908242 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf039b-5a5d-4921-a597-bafc9323ec14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:28.910914 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf039b-5a5d-4921-a597-bafc9323ec14 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:28.917687 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2438afc-eb4c-4ee1-a654-fb2fe1ce5beb None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:28.924207 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a20d1-6df5-4b15-b09f-014695deb0dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:28.924991 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a20d1-6df5-4b15-b09f-014695deb0dd service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:28.925416 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a20d1-6df5-4b15-b09f-014695deb0dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:28.926460 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a20d1-6df5-4b15-b09f-014695deb0dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:28.940175 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a20d1-6df5-4b15-b09f-014695deb0dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:28.945325 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a20d1-6df5-4b15-b09f-014695deb0dd service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:28.960943 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 924/1861] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:28 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:29.065552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 938/1862] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:13:28 2022] POST /identity/v3/projects => generated 373 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:29.068216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a20d1-6df5-4b15-b09f-014695deb0dd service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:29.081666 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf039b-5a5d-4921-a597-bafc9323ec14 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:29.086527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-e9bf039b-5a5d-4921-a597-bafc9323ec14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:29.099682 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a20d1-6df5-4b15-b09f-014695deb0dd service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:29.127170 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=80,keystone:INSERT=5,keystone:DELETE=15 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:29.210285 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f07235-da99-4d17-8c3d-d5bd80a1d3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:29.213965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf039b-5a5d-4921-a597-bafc9323ec14 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:29.224527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a9cef-1cae-4b10-868d-3e20d5e823a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:29.230494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a9cef-1cae-4b10-868d-3e20d5e823a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:29.230494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a9cef-1cae-4b10-868d-3e20d5e823a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:29.230494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a9cef-1cae-4b10-868d-3e20d5e823a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:29.230753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f07235-da99-4d17-8c3d-d5bd80a1d3e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:29.230753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f07235-da99-4d17-8c3d-d5bd80a1d3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:29.230753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f07235-da99-4d17-8c3d-d5bd80a1d3e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:29.230753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f07235-da99-4d17-8c3d-d5bd80a1d3e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:29.231086 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-022a9cef-1cae-4b10-868d-3e20d5e823a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:29.232258 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f07235-da99-4d17-8c3d-d5bd80a1d3e5 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:29.354244 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 925/1863] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:13:29 2022] POST /identity/v3/users => generated 287 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:29.359103 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f07235-da99-4d17-8c3d-d5bd80a1d3e5 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:29.361183 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a9cef-1cae-4b10-868d-3e20d5e823a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:29.364497 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a9cef-1cae-4b10-868d-3e20d5e823a3 service cinder] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:29.376443 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f07235-da99-4d17-8c3d-d5bd80a1d3e5 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:29.382449 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 939/1864] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:29.383053 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a9cef-1cae-4b10-868d-3e20d5e823a3 service cinder] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:29.384379 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-022a9cef-1cae-4b10-868d-3e20d5e823a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:29.387222 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a0cd7-508d-4275-a08c-37d851de11cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:29.388861 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a0cd7-508d-4275-a08c-37d851de11cf admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:29.389157 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a0cd7-508d-4275-a08c-37d851de11cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:29.389433 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a0cd7-508d-4275-a08c-37d851de11cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:29.390141 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334a0cd7-508d-4275-a08c-37d851de11cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:29.391907 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334a0cd7-508d-4275-a08c-37d851de11cf admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:29.406593 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 926/1865] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:29.407525 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a9cef-1cae-4b10-868d-3e20d5e823a3 service cinder] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:29.410493 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a0cd7-508d-4275-a08c-37d851de11cf admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:29.429452 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a0cd7-508d-4275-a08c-37d851de11cf admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:29.432851 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1c4580-d3e1-455d-9409-c8c4d4225446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:29.434394 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c4580-d3e1-455d-9409-c8c4d4225446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:29.435943 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c4580-d3e1-455d-9409-c8c4d4225446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:29.435943 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c4580-d3e1-455d-9409-c8c4d4225446 admin admin] PATH_INFO: `/v3/projects/8be32a432f40442689dcb32fc0450f66` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:29.436245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd45989-014b-4688-9039-c5728169f65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:29.437072 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd45989-014b-4688-9039-c5728169f65a admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:29.437665 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd45989-014b-4688-9039-c5728169f65a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:29.437833 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd45989-014b-4688-9039-c5728169f65a admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:29.438551 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd45989-014b-4688-9039-c5728169f65a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:29.440215 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd45989-014b-4688-9039-c5728169f65a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:29.441821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1c4580-d3e1-455d-9409-c8c4d4225446 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8be32a432f40442689dcb32fc0450f66)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:29.449862 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1c4580-d3e1-455d-9409-c8c4d4225446 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:29.455290 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 927/1866] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:29.459403 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd45989-014b-4688-9039-c5728169f65a admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:29.472927 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd45989-014b-4688-9039-c5728169f65a admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:29.480382 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925e1e0-5c2a-428b-8613-61ae3e479d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:29.481439 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925e1e0-5c2a-428b-8613-61ae3e479d70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:29.481818 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925e1e0-5c2a-428b-8613-61ae3e479d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:29.482239 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925e1e0-5c2a-428b-8613-61ae3e479d70 admin admin] PATH_INFO: `/v3/projects/4c56eb76c3814e488d8447fc18dae874/users/72fea87f31584c6cba9523d52ed7bbd9/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:29.504750 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925e1e0-5c2a-428b-8613-61ae3e479d70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c56eb76c3814e488d8447fc18dae874, user_id=72fea87f31584c6cba9523d52ed7bbd9, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:29.508699 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925e1e0-5c2a-428b-8613-61ae3e479d70 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:29.627040 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 928/1867] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:29 2022] PUT /identity/v3/projects/4c56eb76c3814e488d8447fc18dae874/users/72fea87f31584c6cba9523d52ed7bbd9/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:29.633290 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5223515a-dc29-4b65-9edb-88ced35954ab None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:29.633637 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5223515a-dc29-4b65-9edb-88ced35954ab None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:29.633922 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5223515a-dc29-4b65-9edb-88ced35954ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:29.691512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-5223515a-dc29-4b65-9edb-88ced35954ab None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:29.692779 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-5223515a-dc29-4b65-9edb-88ced35954ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:29.693252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-ef1c4580-d3e1-455d-9409-c8c4d4225446 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8be32a432f40442689dcb32fc0450f66', 'request_id': 'req-ef1c4580-d3e1-455d-9409-c8c4d4225446'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:29.711646 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-ef1c4580-d3e1-455d-9409-c8c4d4225446 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8be32a432f40442689dcb32fc0450f66', 'request_id': 'req-ef1c4580-d3e1-455d-9409-c8c4d4225446'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:29.770985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 929/1868] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:13:29.786835 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5223515a-dc29-4b65-9edb-88ced35954ab None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:29.788817 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-5223515a-dc29-4b65-9edb-88ced35954ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:29.805951 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-ef1c4580-d3e1-455d-9409-c8c4d4225446 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8be32a432f40442689dcb32fc0450f66', 'request_id': 'req-ef1c4580-d3e1-455d-9409-c8c4d4225446'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:29.829465 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-ef1c4580-d3e1-455d-9409-c8c4d4225446 admin admin] The token cache is being invalidate because project 8be32a432f40442689dcb32fc0450f66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:13:29.829465 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-ef1c4580-d3e1-455d-9409-c8c4d4225446 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:29.829465 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-ef1c4580-d3e1-455d-9409-c8c4d4225446 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:29.832054 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 940/1869] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:13:29 2022] DELETE /identity/v3/projects/8be32a432f40442689dcb32fc0450f66 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:29.837014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b1cd1-5b0b-4e6e-984f-459770f671fc None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:29.838886 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b1cd1-5b0b-4e6e-984f-459770f671fc None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:29.839375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b1cd1-5b0b-4e6e-984f-459770f671fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:29.892807 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-fb5b1cd1-5b0b-4e6e-984f-459770f671fc None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:29.896217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-fb5b1cd1-5b0b-4e6e-984f-459770f671fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:29.951163 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 941/1870] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:29.958221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760e9c2-b9a9-4abb-a9de-cef0b144e513 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:29.958821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760e9c2-b9a9-4abb-a9de-cef0b144e513 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:29.959335 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760e9c2-b9a9-4abb-a9de-cef0b144e513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:30.015274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5223515a-dc29-4b65-9edb-88ced35954ab None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:30.029525 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d50075-9652-492a-9d44-9bf1e9846dd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:30.030731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d50075-9652-492a-9d44-9bf1e9846dd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:30.031055 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d50075-9652-492a-9d44-9bf1e9846dd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:30.031380 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d50075-9652-492a-9d44-9bf1e9846dd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:30.039709 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-87d50075-9652-492a-9d44-9bf1e9846dd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:30.045843 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-c760e9c2-b9a9-4abb-a9de-cef0b144e513 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:30.049271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:30.060072 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-c760e9c2-b9a9-4abb-a9de-cef0b144e513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:30.072406 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 942/1871] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:30.074562 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760e9c2-b9a9-4abb-a9de-cef0b144e513 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:30.089855 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d50075-9652-492a-9d44-9bf1e9846dd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:30.093363 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d50075-9652-492a-9d44-9bf1e9846dd3 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:30.123216 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760e9c2-b9a9-4abb-a9de-cef0b144e513 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:30.130223 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 930/1872] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:13:30.145639 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0492c4-2f85-4768-8607-b02b5faf84a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:30.146321 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0492c4-2f85-4768-8607-b02b5faf84a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:30.148859 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0492c4-2f85-4768-8607-b02b5faf84a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:30.149261 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0492c4-2f85-4768-8607-b02b5faf84a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:30.152949 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0492c4-2f85-4768-8607-b02b5faf84a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:13:30.155527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0492c4-2f85-4768-8607-b02b5faf84a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:30.158437 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0492c4-2f85-4768-8607-b02b5faf84a5 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:30.182970 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 943/1873] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:13:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:13:30.184278 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d50075-9652-492a-9d44-9bf1e9846dd3 service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:30.196832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d50075-9652-492a-9d44-9bf1e9846dd3 service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:30.213370 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160327e-ca18-4b85-88e0-2a0573b40333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:30.214348 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160327e-ca18-4b85-88e0-2a0573b40333 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:30.215238 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160327e-ca18-4b85-88e0-2a0573b40333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:30.215715 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160327e-ca18-4b85-88e0-2a0573b40333 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:30.216617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f160327e-ca18-4b85-88e0-2a0573b40333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:30.220982 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f160327e-ca18-4b85-88e0-2a0573b40333 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:30.413525 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 931/1874] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:13:30 2022] POST /identity/v3/projects => generated 385 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:30.415031 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0492c4-2f85-4768-8607-b02b5faf84a5 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:30.435757 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0492c4-2f85-4768-8607-b02b5faf84a5 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:30.563542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb4fc0-7332-49c3-93cd-5b548aac7fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:30.564266 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb4fc0-7332-49c3-93cd-5b548aac7fe6 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:30.564618 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb4fc0-7332-49c3-93cd-5b548aac7fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:30.564964 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb4fc0-7332-49c3-93cd-5b548aac7fe6 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:30.566344 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feb4fc0-7332-49c3-93cd-5b548aac7fe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:30.570530 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feb4fc0-7332-49c3-93cd-5b548aac7fe6 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:30.717181 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 944/1875] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:13:30 2022] POST /identity/v3/users => generated 293 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:30.721447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160327e-ca18-4b85-88e0-2a0573b40333 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:30.734384 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160327e-ca18-4b85-88e0-2a0573b40333 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:30.740930 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc0f9b-cda3-4696-b432-8ab9040da964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:30.741535 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc0f9b-cda3-4696-b432-8ab9040da964 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:30.741876 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc0f9b-cda3-4696-b432-8ab9040da964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:30.742314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc0f9b-cda3-4696-b432-8ab9040da964 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:30.743095 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc0f9b-cda3-4696-b432-8ab9040da964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:30.745763 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc0f9b-cda3-4696-b432-8ab9040da964 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:30.762669 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 932/1876] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:30.764892 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb4fc0-7332-49c3-93cd-5b548aac7fe6 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:30.779179 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb4fc0-7332-49c3-93cd-5b548aac7fe6 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:30.784108 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15e33d0-db19-4889-8b98-14189dd98501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:30.784783 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e33d0-db19-4889-8b98-14189dd98501 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:30.785067 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e33d0-db19-4889-8b98-14189dd98501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:30.785341 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e33d0-db19-4889-8b98-14189dd98501 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:30.786038 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15e33d0-db19-4889-8b98-14189dd98501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:30.787674 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15e33d0-db19-4889-8b98-14189dd98501 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:30.798891 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 945/1877] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:30.803352 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc0f9b-cda3-4696-b432-8ab9040da964 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:30.814500 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc0f9b-cda3-4696-b432-8ab9040da964 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:30.820622 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eef1e8-8ab5-4c41-bc09-b055c362cea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:30.826246 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eef1e8-8ab5-4c41-bc09-b055c362cea1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:30.826246 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eef1e8-8ab5-4c41-bc09-b055c362cea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:30.826246 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eef1e8-8ab5-4c41-bc09-b055c362cea1 admin admin] PATH_INFO: `/v3/projects/2cdc9c3e28ba491faf3616c06fcb3beb/users/805c135b9c7a423a89785f1069f1caec/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:30.838231 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eef1e8-8ab5-4c41-bc09-b055c362cea1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cdc9c3e28ba491faf3616c06fcb3beb, user_id=805c135b9c7a423a89785f1069f1caec, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:30.855164 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eef1e8-8ab5-4c41-bc09-b055c362cea1 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:30.978946 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 933/1878] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:30 2022] PUT /identity/v3/projects/2cdc9c3e28ba491faf3616c06fcb3beb/users/805c135b9c7a423a89785f1069f1caec/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:13:30.982084 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79909f35-a92e-4038-8932-cdc1e1ffa0ec None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:30.982434 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79909f35-a92e-4038-8932-cdc1e1ffa0ec None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:30.982721 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79909f35-a92e-4038-8932-cdc1e1ffa0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:31.032773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-79909f35-a92e-4038-8932-cdc1e1ffa0ec None None] MFA Rules not processed for user `805c135b9c7a423a89785f1069f1caec`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:31.034016 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-79909f35-a92e-4038-8932-cdc1e1ffa0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:31.111190 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 946/1879] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:31.116114 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eef1e8-8ab5-4c41-bc09-b055c362cea1 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:31.184622 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eef1e8-8ab5-4c41-bc09-b055c362cea1 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:31.187362 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab58aa4-39c5-4c73-b602-17e6e4b770e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:31.187987 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab58aa4-39c5-4c73-b602-17e6e4b770e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:31.188274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab58aa4-39c5-4c73-b602-17e6e4b770e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:31.188554 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab58aa4-39c5-4c73-b602-17e6e4b770e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:31.189282 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab58aa4-39c5-4c73-b602-17e6e4b770e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:31.191306 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab58aa4-39c5-4c73-b602-17e6e4b770e3 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:31.329858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 934/1880] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:13:31 2022] POST /identity/v3/projects => generated 381 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:13:31.334035 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79909f35-a92e-4038-8932-cdc1e1ffa0ec None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:31.344597 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79909f35-a92e-4038-8932-cdc1e1ffa0ec None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:31.415613 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e2a4b7-fda8-41bb-adb3-da2f17801d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:31.416399 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2a4b7-fda8-41bb-adb3-da2f17801d17 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:31.416887 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2a4b7-fda8-41bb-adb3-da2f17801d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:31.417291 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2a4b7-fda8-41bb-adb3-da2f17801d17 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:31.418096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e2a4b7-fda8-41bb-adb3-da2f17801d17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:31.420413 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e2a4b7-fda8-41bb-adb3-da2f17801d17 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:31.508917 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 947/1881] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:13:31 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:31.509119 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab58aa4-39c5-4c73-b602-17e6e4b770e3 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:31.525010 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab58aa4-39c5-4c73-b602-17e6e4b770e3 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:31.544743 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6faa3e-4ef4-4c16-b10b-9055f10e1b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:31.545944 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6faa3e-4ef4-4c16-b10b-9055f10e1b65 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:31.546419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6faa3e-4ef4-4c16-b10b-9055f10e1b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:31.546989 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6faa3e-4ef4-4c16-b10b-9055f10e1b65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:31.548200 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6faa3e-4ef4-4c16-b10b-9055f10e1b65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:31.551522 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6faa3e-4ef4-4c16-b10b-9055f10e1b65 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:31.575510 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 935/1882] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:31 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:13:31.579600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e2a4b7-fda8-41bb-adb3-da2f17801d17 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:31.594290 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e2a4b7-fda8-41bb-adb3-da2f17801d17 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:31.604079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21f8755-3c78-4818-89a4-76bbecb4cebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:31.604992 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f8755-3c78-4818-89a4-76bbecb4cebe admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:31.605656 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f8755-3c78-4818-89a4-76bbecb4cebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:31.605934 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f8755-3c78-4818-89a4-76bbecb4cebe admin admin] PATH_INFO: `/v3/projects/1135fd2f6560485da4aa3927a568b37c/users/e562ea6fe897451a96db5475cb2e5794/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:31.642711 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:31.651208 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21f8755-3c78-4818-89a4-76bbecb4cebe admin admin] RBAC: Authorizing `identity:create_grant(project_id=1135fd2f6560485da4aa3927a568b37c, user_id=e562ea6fe897451a96db5475cb2e5794, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:31.653690 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21f8755-3c78-4818-89a4-76bbecb4cebe admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:31.750464 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07006b75-5162-44d5-bf38-28168840e0e0 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:31.752380 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:31.753076 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07006b75-5162-44d5-bf38-28168840e0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:31.753619 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07006b75-5162-44d5-bf38-28168840e0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:31.776235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 948/1883] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:31 2022] PUT /identity/v3/projects/1135fd2f6560485da4aa3927a568b37c/users/e562ea6fe897451a96db5475cb2e5794/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:31.781041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21f8755-3c78-4818-89a4-76bbecb4cebe admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:31.822486 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-07006b75-5162-44d5-bf38-28168840e0e0 None None] MFA Rules not processed for user `72fea87f31584c6cba9523d52ed7bbd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:31.824302 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-07006b75-5162-44d5-bf38-28168840e0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:31.927576 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21f8755-3c78-4818-89a4-76bbecb4cebe admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:31.933825 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 936/1884] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:31.938084 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e800ef-3ceb-429b-bcca-75b962feb541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:31.938736 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e800ef-3ceb-429b-bcca-75b962feb541 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:31.939056 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e800ef-3ceb-429b-bcca-75b962feb541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:31.939457 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07006b75-5162-44d5-bf38-28168840e0e0 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:31.939561 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e800ef-3ceb-429b-bcca-75b962feb541 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:31.940207 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e800ef-3ceb-429b-bcca-75b962feb541 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:31.942450 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e800ef-3ceb-429b-bcca-75b962feb541 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:31.943081 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-07006b75-5162-44d5-bf38-28168840e0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:31.964349 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07006b75-5162-44d5-bf38-28168840e0e0 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:31.972833 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ad7bc-5db7-412b-863c-413e41c702c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:31.974227 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ad7bc-5db7-412b-863c-413e41c702c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:31.974615 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ad7bc-5db7-412b-863c-413e41c702c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:31.978232 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ad7bc-5db7-412b-863c-413e41c702c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:31.978232 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ad7bc-5db7-412b-863c-413e41c702c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:31.983817 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ad7bc-5db7-412b-863c-413e41c702c8 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:31.985686 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 949/1885] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:31 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:31.986655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e800ef-3ceb-429b-bcca-75b962feb541 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:31.998461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e800ef-3ceb-429b-bcca-75b962feb541 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:32.004120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1940d00e-a489-446c-93a6-a12ef125b936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:32.004923 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940d00e-a489-446c-93a6-a12ef125b936 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:32.005326 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940d00e-a489-446c-93a6-a12ef125b936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:32.005702 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940d00e-a489-446c-93a6-a12ef125b936 admin admin] PATH_INFO: `/v3/projects/1135fd2f6560485da4aa3927a568b37c/users/e562ea6fe897451a96db5475cb2e5794/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:32.012673 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1940d00e-a489-446c-93a6-a12ef125b936 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1135fd2f6560485da4aa3927a568b37c, user_id=e562ea6fe897451a96db5475cb2e5794, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:32.024923 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1940d00e-a489-446c-93a6-a12ef125b936 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:32.040370 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 950/1886] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:31 2022] PUT /identity/v3/projects/1135fd2f6560485da4aa3927a568b37c/users/e562ea6fe897451a96db5475cb2e5794/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:32.041812 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b6edf-6a58-4ec6-887b-50c6ae96a546 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:32.042203 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b6edf-6a58-4ec6-887b-50c6ae96a546 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:32.043075 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b6edf-6a58-4ec6-887b-50c6ae96a546 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:32.044482 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 951/1887] 10.222.0.40 () {54 vars in 921 bytes} [Wed Aug 3 05:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:13:32.052682 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aeeffe-1e41-449c-a663-3529d90bee73 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:32.057497 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aeeffe-1e41-449c-a663-3529d90bee73 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:32.057812 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aeeffe-1e41-449c-a663-3529d90bee73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:32.113282 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 937/1888] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:13:31 2022] POST /identity/v3/projects => generated 371 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:32.116037 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c2cb8-d248-43bc-b916-20db7831d2ee None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:32.116341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c2cb8-d248-43bc-b916-20db7831d2ee None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:32.116720 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c2cb8-d248-43bc-b916-20db7831d2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:32.157326 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-55aeeffe-1e41-449c-a663-3529d90bee73 None None] MFA Rules not processed for user `e562ea6fe897451a96db5475cb2e5794`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:32.161243 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-55aeeffe-1e41-449c-a663-3529d90bee73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:32.171619 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-a63c2cb8-d248-43bc-b916-20db7831d2ee None None] MFA Rules not processed for user `a3fa4a6c16274c18849869b55b33ac62`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:32.176675 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a63c2cb8-d248-43bc-b916-20db7831d2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:32.291501 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 952/1889] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:32 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:32.291501 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aeeffe-1e41-449c-a663-3529d90bee73 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:32.359163 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 938/1890] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Aug 3 05:13:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:32.361137 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c2cb8-d248-43bc-b916-20db7831d2ee None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:32.364163 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a63c2cb8-d248-43bc-b916-20db7831d2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:32.444120 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c2cb8-d248-43bc-b916-20db7831d2ee None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:32.449538 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de10d3-8461-4723-a69f-eab0bf592016 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:32.450286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de10d3-8461-4723-a69f-eab0bf592016 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:32.450678 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de10d3-8461-4723-a69f-eab0bf592016 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:32.451084 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de10d3-8461-4723-a69f-eab0bf592016 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:32.463540 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aeeffe-1e41-449c-a663-3529d90bee73 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:32.472010 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bfcd80-57e0-4376-9d3e-a2578bbb8a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:32.475143 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bfcd80-57e0-4376-9d3e-a2578bbb8a83 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:32.475143 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bfcd80-57e0-4376-9d3e-a2578bbb8a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:32.475143 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bfcd80-57e0-4376-9d3e-a2578bbb8a83 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:32.476142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bfcd80-57e0-4376-9d3e-a2578bbb8a83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:32.479354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bfcd80-57e0-4376-9d3e-a2578bbb8a83 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:32.525692 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3de10d3-8461-4723-a69f-eab0bf592016 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:32.527443 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3de10d3-8461-4723-a69f-eab0bf592016 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:32.538874 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 939/1891] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:32.539794 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de10d3-8461-4723-a69f-eab0bf592016 service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:32.541180 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a3de10d3-8461-4723-a69f-eab0bf592016 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:32.560872 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de10d3-8461-4723-a69f-eab0bf592016 service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:32.568569 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c355c83-6edb-47ca-9101-cbb450635a46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:32.569295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c355c83-6edb-47ca-9101-cbb450635a46 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:32.569658 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c355c83-6edb-47ca-9101-cbb450635a46 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:32.570006 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c355c83-6edb-47ca-9101-cbb450635a46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:32.582147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c355c83-6edb-47ca-9101-cbb450635a46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:32.584205 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c355c83-6edb-47ca-9101-cbb450635a46 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:32.592477 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 953/1892] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:13:32 2022] POST /identity/v3/users => generated 292 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:32.595092 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bfcd80-57e0-4376-9d3e-a2578bbb8a83 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:32.599125 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 940/1893] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:32.618910 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bfcd80-57e0-4376-9d3e-a2578bbb8a83 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:32.624731 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4e11f-3089-4d1b-b497-b841365674fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:32.625830 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4e11f-3089-4d1b-b497-b841365674fa admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:32.626205 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4e11f-3089-4d1b-b497-b841365674fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:32.626762 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4e11f-3089-4d1b-b497-b841365674fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:32.627772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e4e11f-3089-4d1b-b497-b841365674fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:32.630634 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e4e11f-3089-4d1b-b497-b841365674fa admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:32.649352 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 954/1894] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:32.650754 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c355c83-6edb-47ca-9101-cbb450635a46 service nova] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:32.670003 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c355c83-6edb-47ca-9101-cbb450635a46 service nova] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:32.680958 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabff8bf-24f3-4ce6-833d-4679968134bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:32.681637 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabff8bf-24f3-4ce6-833d-4679968134bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:32.681925 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabff8bf-24f3-4ce6-833d-4679968134bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:32.682208 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabff8bf-24f3-4ce6-833d-4679968134bb admin admin] PATH_INFO: `/v3/projects/882275428a70495caeab73663243f476/users/234007a4cd97466c894a6cf821856bd9/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:32.727532 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabff8bf-24f3-4ce6-833d-4679968134bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=882275428a70495caeab73663243f476, user_id=234007a4cd97466c894a6cf821856bd9, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:32.730236 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabff8bf-24f3-4ce6-833d-4679968134bb admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:32.878868 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 941/1895] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:32 2022] PUT /identity/v3/projects/882275428a70495caeab73663243f476/users/234007a4cd97466c894a6cf821856bd9/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:32.882812 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4e11f-3089-4d1b-b497-b841365674fa admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:32.897563 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4e11f-3089-4d1b-b497-b841365674fa admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:32.961187 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf84974-afaf-4e10-985f-c2b8b38b0994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:32.965941 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf84974-afaf-4e10-985f-c2b8b38b0994 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:32.965941 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf84974-afaf-4e10-985f-c2b8b38b0994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:32.965941 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf84974-afaf-4e10-985f-c2b8b38b0994 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:32.965941 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf84974-afaf-4e10-985f-c2b8b38b0994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:32.983648 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf84974-afaf-4e10-985f-c2b8b38b0994 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:32.999745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 955/1896] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:33.003967 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabff8bf-24f3-4ce6-833d-4679968134bb admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:33.016944 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabff8bf-24f3-4ce6-833d-4679968134bb admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:33.022806 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e4599-d751-4b3e-8fa4-b1549be01826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:33.025630 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e4599-d751-4b3e-8fa4-b1549be01826 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:33.025630 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e4599-d751-4b3e-8fa4-b1549be01826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:33.025630 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e4599-d751-4b3e-8fa4-b1549be01826 admin admin] PATH_INFO: `/v3/projects/882275428a70495caeab73663243f476/users/234007a4cd97466c894a6cf821856bd9/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:33.026661 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013e4599-d751-4b3e-8fa4-b1549be01826 admin admin] RBAC: Authorizing `identity:create_grant(project_id=882275428a70495caeab73663243f476, user_id=234007a4cd97466c894a6cf821856bd9, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:33.029416 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013e4599-d751-4b3e-8fa4-b1549be01826 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:33.079644 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 942/1897] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:33 2022] PUT /identity/v3/projects/882275428a70495caeab73663243f476/users/234007a4cd97466c894a6cf821856bd9/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:13:33.695353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf84974-afaf-4e10-985f-c2b8b38b0994 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:33.697388 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-1bf84974-afaf-4e10-985f-c2b8b38b0994 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:33.712409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf84974-afaf-4e10-985f-c2b8b38b0994 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:33.798511 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0371ec-56b6-41cd-b597-32f54ccc0715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:33.799220 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0371ec-56b6-41cd-b597-32f54ccc0715 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:33.799681 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0371ec-56b6-41cd-b597-32f54ccc0715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:33.800210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0371ec-56b6-41cd-b597-32f54ccc0715 admin admin] PATH_INFO: `/v3/users/db9f5172b9d14a0585e611d28d50c09e` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:33.801421 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0371ec-56b6-41cd-b597-32f54ccc0715 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db9f5172b9d14a0585e611d28d50c09e)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:33.803443 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0371ec-56b6-41cd-b597-32f54ccc0715 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:34.022087 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-1e0371ec-56b6-41cd-b597-32f54ccc0715 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db9f5172b9d14a0585e611d28d50c09e'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:34.060141 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:34.130264 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-1e0371ec-56b6-41cd-b597-32f54ccc0715 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db9f5172b9d14a0585e611d28d50c09e'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:34.132438 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-1e0371ec-56b6-41cd-b597-32f54ccc0715 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db9f5172b9d14a0585e611d28d50c09e'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:34.150804 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-1e0371ec-56b6-41cd-b597-32f54ccc0715 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db9f5172b9d14a0585e611d28d50c09e'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:34.151828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-1e0371ec-56b6-41cd-b597-32f54ccc0715 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db9f5172b9d14a0585e611d28d50c09e'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:34.170962 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 956/1898] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:13:33 2022] DELETE /identity/v3/users/db9f5172b9d14a0585e611d28d50c09e => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:34.175483 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9d53c-6de6-4a58-9b79-49723808d0c5 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:34.175786 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9d53c-6de6-4a58-9b79-49723808d0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:34.176868 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9d53c-6de6-4a58-9b79-49723808d0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:34.222963 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-6ea9d53c-6de6-4a58-9b79-49723808d0c5 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:34.225119 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6ea9d53c-6de6-4a58-9b79-49723808d0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:34.303623 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 943/1899] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:34.314254 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0371ec-56b6-41cd-b597-32f54ccc0715 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:34.316913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-1e0371ec-56b6-41cd-b597-32f54ccc0715 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:34.396392 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0371ec-56b6-41cd-b597-32f54ccc0715 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:34.400861 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db93f60-6a1d-496e-8b3b-da538e271ec1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:34.401856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db93f60-6a1d-496e-8b3b-da538e271ec1 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:34.402180 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db93f60-6a1d-496e-8b3b-da538e271ec1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:34.402460 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db93f60-6a1d-496e-8b3b-da538e271ec1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:34.412067 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db93f60-6a1d-496e-8b3b-da538e271ec1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:34.413506 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db93f60-6a1d-496e-8b3b-da538e271ec1 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:34.423187 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 957/1900] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:35.045942 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca1552-e2b4-45f3-b402-c50ca5d7dd79 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:35.045942 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca1552-e2b4-45f3-b402-c50ca5d7dd79 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:35.046257 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca1552-e2b4-45f3-b402-c50ca5d7dd79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:35.081348 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-6bca1552-e2b4-45f3-b402-c50ca5d7dd79 None None] MFA Rules not processed for user `234007a4cd97466c894a6cf821856bd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:35.084837 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6bca1552-e2b4-45f3-b402-c50ca5d7dd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:35.157350 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db93f60-6a1d-496e-8b3b-da538e271ec1 service neutron] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:35.160169 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-5db93f60-6a1d-496e-8b3b-da538e271ec1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:35.178956 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db93f60-6a1d-496e-8b3b-da538e271ec1 service neutron] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:35.194102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 944/1901] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:35 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:35.194627 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63b397-d4cd-4914-8bed-b01d868a6bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:35.195364 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63b397-d4cd-4914-8bed-b01d868a6bef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:35.195802 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63b397-d4cd-4914-8bed-b01d868a6bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:35.196091 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63b397-d4cd-4914-8bed-b01d868a6bef admin admin] PATH_INFO: `/v3/projects/fd41573fccb242a2b9277ae51d71dc35` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:35.197852 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe63b397-d4cd-4914-8bed-b01d868a6bef admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd41573fccb242a2b9277ae51d71dc35)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:35.200227 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe63b397-d4cd-4914-8bed-b01d868a6bef admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:35.380764 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-fe63b397-d4cd-4914-8bed-b01d868a6bef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd41573fccb242a2b9277ae51d71dc35', 'request_id': 'req-fe63b397-d4cd-4914-8bed-b01d868a6bef'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:35.383442 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca1552-e2b4-45f3-b402-c50ca5d7dd79 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:35.386069 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6bca1552-e2b4-45f3-b402-c50ca5d7dd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:35.388959 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-fe63b397-d4cd-4914-8bed-b01d868a6bef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd41573fccb242a2b9277ae51d71dc35', 'request_id': 'req-fe63b397-d4cd-4914-8bed-b01d868a6bef'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:35.455772 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca1552-e2b4-45f3-b402-c50ca5d7dd79 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:35.515173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96661a2b-b8da-44a5-8f90-abbcdf4f8095 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:35.515173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96661a2b-b8da-44a5-8f90-abbcdf4f8095 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:35.515173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96661a2b-b8da-44a5-8f90-abbcdf4f8095 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:35.515173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96661a2b-b8da-44a5-8f90-abbcdf4f8095 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:35.516281 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-fe63b397-d4cd-4914-8bed-b01d868a6bef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd41573fccb242a2b9277ae51d71dc35', 'request_id': 'req-fe63b397-d4cd-4914-8bed-b01d868a6bef'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:35.517896 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-96661a2b-b8da-44a5-8f90-abbcdf4f8095 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:35.525815 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:35.545073 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-fe63b397-d4cd-4914-8bed-b01d868a6bef admin admin] The token cache is being invalidate because project fd41573fccb242a2b9277ae51d71dc35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:13:35.546800 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-fe63b397-d4cd-4914-8bed-b01d868a6bef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:35.546800 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-fe63b397-d4cd-4914-8bed-b01d868a6bef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:35.548780 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 958/1902] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:13:35 2022] DELETE /identity/v3/projects/fd41573fccb242a2b9277ae51d71dc35 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:13:35.551935 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63b397-d4cd-4914-8bed-b01d868a6bef admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:35.553623 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-fe63b397-d4cd-4914-8bed-b01d868a6bef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:35.622610 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96661a2b-b8da-44a5-8f90-abbcdf4f8095 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:35.624189 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96661a2b-b8da-44a5-8f90-abbcdf4f8095 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:35.636590 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 945/1903] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:13:35.691648 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63b397-d4cd-4914-8bed-b01d868a6bef admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:35.698635 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19833a1b-a65f-4963-a7a5-4fd848abefff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:35.699579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19833a1b-a65f-4963-a7a5-4fd848abefff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:35.700097 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19833a1b-a65f-4963-a7a5-4fd848abefff admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:35.700496 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19833a1b-a65f-4963-a7a5-4fd848abefff admin admin] PATH_INFO: `/v3/users/c4ca3ca7812e4413bca591321e2e1b2a` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:35.701769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19833a1b-a65f-4963-a7a5-4fd848abefff admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4ca3ca7812e4413bca591321e2e1b2a)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:35.703958 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19833a1b-a65f-4963-a7a5-4fd848abefff admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:35.911477 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-19833a1b-a65f-4963-a7a5-4fd848abefff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4ca3ca7812e4413bca591321e2e1b2a'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:35.965209 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96661a2b-b8da-44a5-8f90-abbcdf4f8095 service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:35.967581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-96661a2b-b8da-44a5-8f90-abbcdf4f8095 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:35.970108 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-19833a1b-a65f-4963-a7a5-4fd848abefff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4ca3ca7812e4413bca591321e2e1b2a'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:35.972035 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-19833a1b-a65f-4963-a7a5-4fd848abefff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4ca3ca7812e4413bca591321e2e1b2a'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:35.990877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-19833a1b-a65f-4963-a7a5-4fd848abefff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4ca3ca7812e4413bca591321e2e1b2a'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:35.991649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-19833a1b-a65f-4963-a7a5-4fd848abefff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4ca3ca7812e4413bca591321e2e1b2a'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:35.993098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96661a2b-b8da-44a5-8f90-abbcdf4f8095 service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:36.005860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 959/1904] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:13:35 2022] DELETE /identity/v3/users/c4ca3ca7812e4413bca591321e2e1b2a => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:13:36.008360 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda4bde-02a4-4643-b133-fccfb4b2e3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:36.015308 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda4bde-02a4-4643-b133-fccfb4b2e3f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:36.015975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda4bde-02a4-4643-b133-fccfb4b2e3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:36.019953 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda4bde-02a4-4643-b133-fccfb4b2e3f8 admin admin] PATH_INFO: `/v3/users/16a650a612ea4d3495e28ba33686b709` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:36.020415 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcda4bde-02a4-4643-b133-fccfb4b2e3f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16a650a612ea4d3495e28ba33686b709)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:36.023848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcda4bde-02a4-4643-b133-fccfb4b2e3f8 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:36.294290 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-dcda4bde-02a4-4643-b133-fccfb4b2e3f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16a650a612ea4d3495e28ba33686b709'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:36.396933 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-dcda4bde-02a4-4643-b133-fccfb4b2e3f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16a650a612ea4d3495e28ba33686b709'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:36.397635 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-dcda4bde-02a4-4643-b133-fccfb4b2e3f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16a650a612ea4d3495e28ba33686b709'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:36.413543 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-dcda4bde-02a4-4643-b133-fccfb4b2e3f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16a650a612ea4d3495e28ba33686b709'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:36.414276 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-dcda4bde-02a4-4643-b133-fccfb4b2e3f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16a650a612ea4d3495e28ba33686b709'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:36.431822 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 946/1905] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:13:35 2022] DELETE /identity/v3/users/16a650a612ea4d3495e28ba33686b709 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:36.432107 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7faf76-1fc0-492b-bde8-687e2dc3c160 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:36.432581 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7faf76-1fc0-492b-bde8-687e2dc3c160 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:36.432885 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7faf76-1fc0-492b-bde8-687e2dc3c160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:36.460141 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-cc7faf76-1fc0-492b-bde8-687e2dc3c160 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:36.461462 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-cc7faf76-1fc0-492b-bde8-687e2dc3c160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:36.535231 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 960/1906] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:13:36.545246 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda4bde-02a4-4643-b133-fccfb4b2e3f8 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:36.547360 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-dcda4bde-02a4-4643-b133-fccfb4b2e3f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:36.620236 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7faf76-1fc0-492b-bde8-687e2dc3c160 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:36.622231 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-cc7faf76-1fc0-492b-bde8-687e2dc3c160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:36.641813 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7faf76-1fc0-492b-bde8-687e2dc3c160 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:36.647899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda4bde-02a4-4643-b133-fccfb4b2e3f8 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:36.653673 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185b8d5-8b40-45df-afed-dcf98b3c4439 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:36.654512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185b8d5-8b40-45df-afed-dcf98b3c4439 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:36.654809 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185b8d5-8b40-45df-afed-dcf98b3c4439 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:36.655132 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185b8d5-8b40-45df-afed-dcf98b3c4439 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:36.657743 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d64e9ad-2cc4-46eb-8581-4060ab5488b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:36.658556 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64e9ad-2cc4-46eb-8581-4060ab5488b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:36.659081 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64e9ad-2cc4-46eb-8581-4060ab5488b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:36.659461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64e9ad-2cc4-46eb-8581-4060ab5488b2 admin admin] PATH_INFO: `/v3/projects/f242d01b08704c189d53de46334308a1` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:36.660844 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d64e9ad-2cc4-46eb-8581-4060ab5488b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f242d01b08704c189d53de46334308a1)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:36.664482 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d64e9ad-2cc4-46eb-8581-4060ab5488b2 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:36.681135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185b8d5-8b40-45df-afed-dcf98b3c4439 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:36.681135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185b8d5-8b40-45df-afed-dcf98b3c4439 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:36.693514 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 947/1907] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:36.872210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-0d64e9ad-2cc4-46eb-8581-4060ab5488b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f242d01b08704c189d53de46334308a1', 'request_id': 'req-0d64e9ad-2cc4-46eb-8581-4060ab5488b2'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:36.880023 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-0d64e9ad-2cc4-46eb-8581-4060ab5488b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f242d01b08704c189d53de46334308a1', 'request_id': 'req-0d64e9ad-2cc4-46eb-8581-4060ab5488b2'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:36.987623 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-0d64e9ad-2cc4-46eb-8581-4060ab5488b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f242d01b08704c189d53de46334308a1', 'request_id': 'req-0d64e9ad-2cc4-46eb-8581-4060ab5488b2'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:37.002063 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-0d64e9ad-2cc4-46eb-8581-4060ab5488b2 admin admin] The token cache is being invalidate because project f242d01b08704c189d53de46334308a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:13:37.002758 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-0d64e9ad-2cc4-46eb-8581-4060ab5488b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:37.003747 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-0d64e9ad-2cc4-46eb-8581-4060ab5488b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:37.006010 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 961/1908] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:13:36 2022] DELETE /identity/v3/projects/f242d01b08704c189d53de46334308a1 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:13:37.010951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9467557-2030-468d-b059-36a97540ce4c None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:37.011682 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9467557-2030-468d-b059-36a97540ce4c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:37.012156 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9467557-2030-468d-b059-36a97540ce4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:37.047225 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-c9467557-2030-468d-b059-36a97540ce4c None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:37.051393 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c9467557-2030-468d-b059-36a97540ce4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:37.134767 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 948/1909] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:37.137370 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1189758-9338-4862-9774-d5e2d1155189 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:37.137785 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1189758-9338-4862-9774-d5e2d1155189 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:37.138161 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1189758-9338-4862-9774-d5e2d1155189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:37.147466 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:37.188944 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-c1189758-9338-4862-9774-d5e2d1155189 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:37.191094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-c1189758-9338-4862-9774-d5e2d1155189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:37.207549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 962/1910] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:13:37.208663 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9467557-2030-468d-b059-36a97540ce4c None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:37.218935 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9467557-2030-468d-b059-36a97540ce4c None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:37.224924 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d48a88-b073-481b-9b70-b0c50ba5c889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:37.225623 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d48a88-b073-481b-9b70-b0c50ba5c889 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:37.225906 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d48a88-b073-481b-9b70-b0c50ba5c889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:37.226181 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d48a88-b073-481b-9b70-b0c50ba5c889 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:37.226922 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d48a88-b073-481b-9b70-b0c50ba5c889 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:13:37.227231 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d48a88-b073-481b-9b70-b0c50ba5c889 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:37.228867 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d48a88-b073-481b-9b70-b0c50ba5c889 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:37.242521 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 949/1911] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:13:37.248731 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1189758-9338-4862-9774-d5e2d1155189 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:37.259507 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1189758-9338-4862-9774-d5e2d1155189 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:37.264815 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d1fb1-b045-4462-a6cb-aca952c7d3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:37.265768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d1fb1-b045-4462-a6cb-aca952c7d3b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:37.266184 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d1fb1-b045-4462-a6cb-aca952c7d3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:37.266557 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d1fb1-b045-4462-a6cb-aca952c7d3b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:37.267427 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212d1fb1-b045-4462-a6cb-aca952c7d3b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:37.270194 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212d1fb1-b045-4462-a6cb-aca952c7d3b4 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:37.416692 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 963/1912] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:13:37 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:13:37.420863 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d48a88-b073-481b-9b70-b0c50ba5c889 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:37.433782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d48a88-b073-481b-9b70-b0c50ba5c889 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:37.462390 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d1fb1-b045-4462-a6cb-aca952c7d3b4 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:37.464779 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-212d1fb1-b045-4462-a6cb-aca952c7d3b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:37.560038 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe47e02-dada-4703-85c6-31218d4b4cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:37.560801 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe47e02-dada-4703-85c6-31218d4b4cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:37.561207 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe47e02-dada-4703-85c6-31218d4b4cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:37.561572 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe47e02-dada-4703-85c6-31218d4b4cb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:37.562270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe47e02-dada-4703-85c6-31218d4b4cb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:37.564445 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe47e02-dada-4703-85c6-31218d4b4cb7 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:37.567241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d1fb1-b045-4462-a6cb-aca952c7d3b4 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:37.572726 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf553d5-301c-4fcd-8f59-1c5eae137a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:37.573612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf553d5-301c-4fcd-8f59-1c5eae137a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:37.574143 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf553d5-301c-4fcd-8f59-1c5eae137a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:37.574522 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf553d5-301c-4fcd-8f59-1c5eae137a9e admin admin] PATH_INFO: `/v3/projects/2fd8d36a8f474a2ea2fd04edf013b2a8` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:37.575924 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf553d5-301c-4fcd-8f59-1c5eae137a9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fd8d36a8f474a2ea2fd04edf013b2a8)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:37.578020 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf553d5-301c-4fcd-8f59-1c5eae137a9e admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:37.674143 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 950/1913] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:13:37 2022] POST /identity/v3/users => generated 293 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:13:37.675179 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe47e02-dada-4703-85c6-31218d4b4cb7 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:37.678507 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-4fe47e02-dada-4703-85c6-31218d4b4cb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:37.742739 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe47e02-dada-4703-85c6-31218d4b4cb7 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:37.749172 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-faf553d5-301c-4fcd-8f59-1c5eae137a9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fd8d36a8f474a2ea2fd04edf013b2a8', 'request_id': 'req-faf553d5-301c-4fcd-8f59-1c5eae137a9e'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:37.757649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-faf553d5-301c-4fcd-8f59-1c5eae137a9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fd8d36a8f474a2ea2fd04edf013b2a8', 'request_id': 'req-faf553d5-301c-4fcd-8f59-1c5eae137a9e'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:37.804092 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c67ce-9d45-483f-955d-051839dcbb3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:37.804719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c67ce-9d45-483f-955d-051839dcbb3b service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:37.805005 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c67ce-9d45-483f-955d-051839dcbb3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:37.805281 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c67ce-9d45-483f-955d-051839dcbb3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:37.807054 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-eb1c67ce-9d45-483f-955d-051839dcbb3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:37.861706 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-faf553d5-301c-4fcd-8f59-1c5eae137a9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fd8d36a8f474a2ea2fd04edf013b2a8', 'request_id': 'req-faf553d5-301c-4fcd-8f59-1c5eae137a9e'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:37.882511 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c67ce-9d45-483f-955d-051839dcbb3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:37.884267 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c67ce-9d45-483f-955d-051839dcbb3b service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:37.885744 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-faf553d5-301c-4fcd-8f59-1c5eae137a9e admin admin] The token cache is being invalidate because project 2fd8d36a8f474a2ea2fd04edf013b2a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:13:37.886195 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-faf553d5-301c-4fcd-8f59-1c5eae137a9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:37.888468 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-faf553d5-301c-4fcd-8f59-1c5eae137a9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:37.890834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 964/1914] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:13:37 2022] DELETE /identity/v3/projects/2fd8d36a8f474a2ea2fd04edf013b2a8 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:13:37.895440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf553d5-301c-4fcd-8f59-1c5eae137a9e admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:37.897145 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-faf553d5-301c-4fcd-8f59-1c5eae137a9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:37.905041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 951/1915] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:37 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:37.906913 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c67ce-9d45-483f-955d-051839dcbb3b service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:37.991537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c67ce-9d45-483f-955d-051839dcbb3b service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:38.001626 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32493613-b5a7-4c19-90bf-a551209eec92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:38.002303 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32493613-b5a7-4c19-90bf-a551209eec92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:38.002929 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32493613-b5a7-4c19-90bf-a551209eec92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:38.003408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32493613-b5a7-4c19-90bf-a551209eec92 admin admin] PATH_INFO: `/v3/users/5d2be48511224fc5aa1dd7e92c54b7e9` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:38.004664 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32493613-b5a7-4c19-90bf-a551209eec92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d2be48511224fc5aa1dd7e92c54b7e9)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:38.006802 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32493613-b5a7-4c19-90bf-a551209eec92 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:38.008600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf553d5-301c-4fcd-8f59-1c5eae137a9e admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:38.015890 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57618b2f-3225-4bfc-a046-025c8b7ad26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:38.017779 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57618b2f-3225-4bfc-a046-025c8b7ad26c admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:38.017779 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57618b2f-3225-4bfc-a046-025c8b7ad26c admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:38.017779 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57618b2f-3225-4bfc-a046-025c8b7ad26c admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:38.018614 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57618b2f-3225-4bfc-a046-025c8b7ad26c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:38.020746 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57618b2f-3225-4bfc-a046-025c8b7ad26c admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:38.038126 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 965/1916] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:38.045002 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57618b2f-3225-4bfc-a046-025c8b7ad26c admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:38.061075 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57618b2f-3225-4bfc-a046-025c8b7ad26c admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:38.068182 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac6d94-6ed9-407d-8629-39ba3cc32268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:38.068872 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac6d94-6ed9-407d-8629-39ba3cc32268 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:38.069158 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac6d94-6ed9-407d-8629-39ba3cc32268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:38.069438 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac6d94-6ed9-407d-8629-39ba3cc32268 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:38.070145 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac6d94-6ed9-407d-8629-39ba3cc32268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:38.072074 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac6d94-6ed9-407d-8629-39ba3cc32268 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:38.088457 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 966/1917] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:38.092471 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac6d94-6ed9-407d-8629-39ba3cc32268 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:38.126663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac6d94-6ed9-407d-8629-39ba3cc32268 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:38.132124 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a44753c-63ea-4cbf-87dd-2ba81f4d63a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:38.133343 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44753c-63ea-4cbf-87dd-2ba81f4d63a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:38.133857 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44753c-63ea-4cbf-87dd-2ba81f4d63a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:38.134038 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44753c-63ea-4cbf-87dd-2ba81f4d63a8 admin admin] PATH_INFO: `/v3/projects/25af358784094e94b201d1f7e49a4b3a/users/9eea6184cd4041b0802145f47c0be491/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:38.173843 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a44753c-63ea-4cbf-87dd-2ba81f4d63a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25af358784094e94b201d1f7e49a4b3a, user_id=9eea6184cd4041b0802145f47c0be491, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:38.179221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a44753c-63ea-4cbf-87dd-2ba81f4d63a8 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:38.285453 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-32493613-b5a7-4c19-90bf-a551209eec92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d2be48511224fc5aa1dd7e92c54b7e9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:38.306222 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 967/1918] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:38 2022] PUT /identity/v3/projects/25af358784094e94b201d1f7e49a4b3a/users/9eea6184cd4041b0802145f47c0be491/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:38.316785 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505740e-5ec8-4854-a9d9-717c0a4d03cc None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:38.317094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505740e-5ec8-4854-a9d9-717c0a4d03cc None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:38.317382 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505740e-5ec8-4854-a9d9-717c0a4d03cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:38.349727 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-3505740e-5ec8-4854-a9d9-717c0a4d03cc None None] MFA Rules not processed for user `9eea6184cd4041b0802145f47c0be491`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:38.351026 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-3505740e-5ec8-4854-a9d9-717c0a4d03cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:38.425242 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 968/1919] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:13:38.427941 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-32493613-b5a7-4c19-90bf-a551209eec92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d2be48511224fc5aa1dd7e92c54b7e9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:38.429633 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-32493613-b5a7-4c19-90bf-a551209eec92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d2be48511224fc5aa1dd7e92c54b7e9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:38.430100 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505740e-5ec8-4854-a9d9-717c0a4d03cc None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:38.455800 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-32493613-b5a7-4c19-90bf-a551209eec92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d2be48511224fc5aa1dd7e92c54b7e9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:38.457026 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-32493613-b5a7-4c19-90bf-a551209eec92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d2be48511224fc5aa1dd7e92c54b7e9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:38.473974 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 952/1920] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:13:37 2022] DELETE /identity/v3/users/5d2be48511224fc5aa1dd7e92c54b7e9 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:38.534872 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505740e-5ec8-4854-a9d9-717c0a4d03cc None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:38.537040 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:38.614148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dab117-9e70-4561-b6ec-9c25809ed081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:38.614908 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dab117-9e70-4561-b6ec-9c25809ed081 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:38.615232 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dab117-9e70-4561-b6ec-9c25809ed081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:38.615518 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dab117-9e70-4561-b6ec-9c25809ed081 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:38.616206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dab117-9e70-4561-b6ec-9c25809ed081 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:38.618206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dab117-9e70-4561-b6ec-9c25809ed081 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:38.739408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 969/1921] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:13:38 2022] POST /identity/v3/projects => generated 387 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:38.739993 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32493613-b5a7-4c19-90bf-a551209eec92 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:38.742472 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-32493613-b5a7-4c19-90bf-a551209eec92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:38.751066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dab117-9e70-4561-b6ec-9c25809ed081 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:38.754258 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-b7dab117-9e70-4561-b6ec-9c25809ed081 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:38.759555 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32493613-b5a7-4c19-90bf-a551209eec92 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:38.833517 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33195c30-2c60-4183-bc69-14b8349db23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:38.836832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33195c30-2c60-4183-bc69-14b8349db23c admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:38.837124 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dab117-9e70-4561-b6ec-9c25809ed081 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:38.837254 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33195c30-2c60-4183-bc69-14b8349db23c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:38.837414 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33195c30-2c60-4183-bc69-14b8349db23c admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:38.838087 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33195c30-2c60-4183-bc69-14b8349db23c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:38.840112 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33195c30-2c60-4183-bc69-14b8349db23c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:38.841339 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4864f-473d-444a-9894-ca40ae3f07be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:38.842362 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4864f-473d-444a-9894-ca40ae3f07be service cinder] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:38.842362 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4864f-473d-444a-9894-ca40ae3f07be service cinder] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:38.842879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4864f-473d-444a-9894-ca40ae3f07be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:38.844848 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-a8c4864f-473d-444a-9894-ca40ae3f07be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:38.942732 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 953/1922] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:13:38 2022] POST /identity/v3/users => generated 300 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:38.943853 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33195c30-2c60-4183-bc69-14b8349db23c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:38.955934 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33195c30-2c60-4183-bc69-14b8349db23c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:38.963082 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88c1e6-f80a-4177-bfda-0ce5a1c0fa13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:38.963980 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88c1e6-f80a-4177-bfda-0ce5a1c0fa13 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:38.964419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88c1e6-f80a-4177-bfda-0ce5a1c0fa13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:38.965290 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88c1e6-f80a-4177-bfda-0ce5a1c0fa13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:38.966180 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b88c1e6-f80a-4177-bfda-0ce5a1c0fa13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:38.968308 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b88c1e6-f80a-4177-bfda-0ce5a1c0fa13 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:38.977111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c4864f-473d-444a-9894-ca40ae3f07be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:38.979524 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c4864f-473d-444a-9894-ca40ae3f07be service cinder] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:38.986446 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 954/1923] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:38.988497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88c1e6-f80a-4177-bfda-0ce5a1c0fa13 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:38.998570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 970/1924] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:38 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:13:39.001934 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88c1e6-f80a-4177-bfda-0ce5a1c0fa13 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:39.008267 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db302f0b-b144-4de8-bd81-2ca27a67ee4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:39.008971 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db302f0b-b144-4de8-bd81-2ca27a67ee4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:39.009258 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db302f0b-b144-4de8-bd81-2ca27a67ee4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:39.009712 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db302f0b-b144-4de8-bd81-2ca27a67ee4a admin admin] PATH_INFO: `/v3/projects/63f33ed61f004c6cb170c85a56f6cdeb/users/13ffc8bb62014c3094e0b71f9a57ba3d/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:39.028092 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db302f0b-b144-4de8-bd81-2ca27a67ee4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=63f33ed61f004c6cb170c85a56f6cdeb, user_id=13ffc8bb62014c3094e0b71f9a57ba3d, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:39.031498 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db302f0b-b144-4de8-bd81-2ca27a67ee4a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:39.136450 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 955/1925] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:38 2022] PUT /identity/v3/projects/63f33ed61f004c6cb170c85a56f6cdeb/users/13ffc8bb62014c3094e0b71f9a57ba3d/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:39.142946 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4864f-473d-444a-9894-ca40ae3f07be service cinder] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:39.142946 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-a8c4864f-473d-444a-9894-ca40ae3f07be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:39.143692 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db302f0b-b144-4de8-bd81-2ca27a67ee4a admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:39.171464 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4864f-473d-444a-9894-ca40ae3f07be service cinder] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:39.254067 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea12f6-6caf-4f1d-8459-8a52cc09615e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:39.255538 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea12f6-6caf-4f1d-8459-8a52cc09615e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:39.255814 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea12f6-6caf-4f1d-8459-8a52cc09615e admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:39.256103 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea12f6-6caf-4f1d-8459-8a52cc09615e admin admin] PATH_INFO: `/v3/projects/264942a659dc488fa1bc2cf11fc1eda9` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:39.257440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea12f6-6caf-4f1d-8459-8a52cc09615e admin admin] RBAC: Authorizing `identity:delete_project(project_id=264942a659dc488fa1bc2cf11fc1eda9)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:39.260514 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea12f6-6caf-4f1d-8459-8a52cc09615e admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:39.267084 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db302f0b-b144-4de8-bd81-2ca27a67ee4a admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:39.280447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4528eb-8655-467c-8884-c157d7ae107c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:39.281097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4528eb-8655-467c-8884-c157d7ae107c admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:39.281385 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4528eb-8655-467c-8884-c157d7ae107c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:39.281661 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4528eb-8655-467c-8884-c157d7ae107c admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:39.282483 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4528eb-8655-467c-8884-c157d7ae107c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:39.284884 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4528eb-8655-467c-8884-c157d7ae107c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:39.298889 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 956/1926] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:13:39.304457 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4528eb-8655-467c-8884-c157d7ae107c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:39.318890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4528eb-8655-467c-8884-c157d7ae107c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:39.325047 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda3582-98fe-4c59-a43c-a9028506f986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:39.325911 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda3582-98fe-4c59-a43c-a9028506f986 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:39.326263 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda3582-98fe-4c59-a43c-a9028506f986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:39.326659 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda3582-98fe-4c59-a43c-a9028506f986 admin admin] PATH_INFO: `/v3/projects/63f33ed61f004c6cb170c85a56f6cdeb/users/13ffc8bb62014c3094e0b71f9a57ba3d/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:39.328859 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bda3582-98fe-4c59-a43c-a9028506f986 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63f33ed61f004c6cb170c85a56f6cdeb, user_id=13ffc8bb62014c3094e0b71f9a57ba3d, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:39.335329 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bda3582-98fe-4c59-a43c-a9028506f986 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:39.344542 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 957/1927] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:39 2022] PUT /identity/v3/projects/63f33ed61f004c6cb170c85a56f6cdeb/users/13ffc8bb62014c3094e0b71f9a57ba3d/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:13:39.367571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35584814-d4eb-43d0-86d8-0ec57fee5348 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:39.368807 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35584814-d4eb-43d0-86d8-0ec57fee5348 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:39.369467 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35584814-d4eb-43d0-86d8-0ec57fee5348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:39.432242 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-35584814-d4eb-43d0-86d8-0ec57fee5348 None None] MFA Rules not processed for user `13ffc8bb62014c3094e0b71f9a57ba3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:39.435208 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-35584814-d4eb-43d0-86d8-0ec57fee5348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:39.437269 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-f6ea12f6-6caf-4f1d-8459-8a52cc09615e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '264942a659dc488fa1bc2cf11fc1eda9', 'request_id': 'req-f6ea12f6-6caf-4f1d-8459-8a52cc09615e'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:39.446693 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-f6ea12f6-6caf-4f1d-8459-8a52cc09615e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '264942a659dc488fa1bc2cf11fc1eda9', 'request_id': 'req-f6ea12f6-6caf-4f1d-8459-8a52cc09615e'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:39.526289 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 958/1928] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:39 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:39.537562 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-f6ea12f6-6caf-4f1d-8459-8a52cc09615e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '264942a659dc488fa1bc2cf11fc1eda9', 'request_id': 'req-f6ea12f6-6caf-4f1d-8459-8a52cc09615e'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:39.552212 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-f6ea12f6-6caf-4f1d-8459-8a52cc09615e admin admin] The token cache is being invalidate because project 264942a659dc488fa1bc2cf11fc1eda9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:13:39.552531 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-f6ea12f6-6caf-4f1d-8459-8a52cc09615e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:39.553201 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-f6ea12f6-6caf-4f1d-8459-8a52cc09615e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:39.554750 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 971/1929] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:13:39 2022] DELETE /identity/v3/projects/264942a659dc488fa1bc2cf11fc1eda9 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:39.559706 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d5db8-2b35-4abb-b351-b1017a2e3707 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:39.560025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d5db8-2b35-4abb-b351-b1017a2e3707 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:39.560372 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d5db8-2b35-4abb-b351-b1017a2e3707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:39.579866 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-f10d5db8-2b35-4abb-b351-b1017a2e3707 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:39.581118 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f10d5db8-2b35-4abb-b351-b1017a2e3707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:39.651798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 959/1930] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:39.666860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274463a-b198-4341-8ea5-b696df6cd072 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:39.666860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274463a-b198-4341-8ea5-b696df6cd072 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:39.666860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274463a-b198-4341-8ea5-b696df6cd072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:39.682201 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d5db8-2b35-4abb-b351-b1017a2e3707 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:39.684448 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f10d5db8-2b35-4abb-b351-b1017a2e3707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:39.691103 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-5274463a-b198-4341-8ea5-b696df6cd072 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:39.692373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-5274463a-b198-4341-8ea5-b696df6cd072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:39.700542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 972/1931] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:39.704976 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5274463a-b198-4341-8ea5-b696df6cd072 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:39.715299 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5274463a-b198-4341-8ea5-b696df6cd072 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:39.719981 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4378daa-6764-4cfb-a339-620aa3cd3835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:39.720628 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4378daa-6764-4cfb-a339-620aa3cd3835 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:39.720902 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4378daa-6764-4cfb-a339-620aa3cd3835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:39.721176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4378daa-6764-4cfb-a339-620aa3cd3835 admin admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:39.722397 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4378daa-6764-4cfb-a339-620aa3cd3835 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:13:39.722702 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4378daa-6764-4cfb-a339-620aa3cd3835 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:39.724603 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4378daa-6764-4cfb-a339-620aa3cd3835 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:39.739954 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 973/1932] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:13:39.744390 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4378daa-6764-4cfb-a339-620aa3cd3835 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:39.753858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d5db8-2b35-4abb-b351-b1017a2e3707 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:39.756263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4378daa-6764-4cfb-a339-620aa3cd3835 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:39.758209 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c1a38-1a1f-47a5-b46d-869896549b9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:39.758859 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c1a38-1a1f-47a5-b46d-869896549b9d service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:39.759201 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c1a38-1a1f-47a5-b46d-869896549b9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:39.759439 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c1a38-1a1f-47a5-b46d-869896549b9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:39.761247 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-118c1a38-1a1f-47a5-b46d-869896549b9d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:39.762336 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a15e4a-c2b8-49dc-9c60-7721a1e328ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:39.763154 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a15e4a-c2b8-49dc-9c60-7721a1e328ca admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:39.764437 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a15e4a-c2b8-49dc-9c60-7721a1e328ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:39.764437 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a15e4a-c2b8-49dc-9c60-7721a1e328ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:39.764640 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a15e4a-c2b8-49dc-9c60-7721a1e328ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:39.766516 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a15e4a-c2b8-49dc-9c60-7721a1e328ca admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:39.790181 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c1a38-1a1f-47a5-b46d-869896549b9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:39.792413 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c1a38-1a1f-47a5-b46d-869896549b9d service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:39.806034 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 960/1933] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:39 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:13:39.840183 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c1a38-1a1f-47a5-b46d-869896549b9d service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:39.842015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-118c1a38-1a1f-47a5-b46d-869896549b9d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:39.859830 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c1a38-1a1f-47a5-b46d-869896549b9d service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:39.867049 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13b991-bf54-4400-a2c2-e5be10425a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:39.867892 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13b991-bf54-4400-a2c2-e5be10425a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:39.868706 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13b991-bf54-4400-a2c2-e5be10425a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:39.868706 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13b991-bf54-4400-a2c2-e5be10425a37 admin admin] PATH_INFO: `/v3/users/9eea6184cd4041b0802145f47c0be491` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:39.870865 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb13b991-bf54-4400-a2c2-e5be10425a37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eea6184cd4041b0802145f47c0be491)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:39.873256 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb13b991-bf54-4400-a2c2-e5be10425a37 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:39.904960 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 974/1934] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:13:39 2022] POST /identity/v3/projects => generated 409 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:13:39.909610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a15e4a-c2b8-49dc-9c60-7721a1e328ca admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:39.990797 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a15e4a-c2b8-49dc-9c60-7721a1e328ca admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:39.995941 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c710b-17cc-4fe6-a257-60f51f4fdae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:39.996563 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c710b-17cc-4fe6-a257-60f51f4fdae9 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:39.996841 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c710b-17cc-4fe6-a257-60f51f4fdae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:39.997119 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c710b-17cc-4fe6-a257-60f51f4fdae9 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:39.997816 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12c710b-17cc-4fe6-a257-60f51f4fdae9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:39.999784 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12c710b-17cc-4fe6-a257-60f51f4fdae9 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:40.050218 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-eb13b991-bf54-4400-a2c2-e5be10425a37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eea6184cd4041b0802145f47c0be491'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:40.130570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-eb13b991-bf54-4400-a2c2-e5be10425a37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eea6184cd4041b0802145f47c0be491'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:40.131647 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-eb13b991-bf54-4400-a2c2-e5be10425a37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eea6184cd4041b0802145f47c0be491'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:40.136659 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 975/1935] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:13:39 2022] POST /identity/v3/users => generated 305 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:40.142274 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c710b-17cc-4fe6-a257-60f51f4fdae9 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:40.147279 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-eb13b991-bf54-4400-a2c2-e5be10425a37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eea6184cd4041b0802145f47c0be491'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:40.148072 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-eb13b991-bf54-4400-a2c2-e5be10425a37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eea6184cd4041b0802145f47c0be491'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:40.153239 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c710b-17cc-4fe6-a257-60f51f4fdae9 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:40.159973 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a5385-1de7-439b-8164-ff648741db34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:40.160763 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a5385-1de7-439b-8164-ff648741db34 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:40.161197 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a5385-1de7-439b-8164-ff648741db34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:40.161612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a5385-1de7-439b-8164-ff648741db34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:40.162320 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 961/1936] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:13:39 2022] DELETE /identity/v3/users/9eea6184cd4041b0802145f47c0be491 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:13:40.162583 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a5385-1de7-439b-8164-ff648741db34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:40.164389 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a5385-1de7-439b-8164-ff648741db34 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:40.166645 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1300c67-b2af-4423-a62d-dcce0992fe27 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:40.166975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1300c67-b2af-4423-a62d-dcce0992fe27 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:40.167281 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1300c67-b2af-4423-a62d-dcce0992fe27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:40.178574 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 976/1937] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:13:40.181525 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a5385-1de7-439b-8164-ff648741db34 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:40.183921 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-730a5385-1de7-439b-8164-ff648741db34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:40.187890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-c1300c67-b2af-4423-a62d-dcce0992fe27 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:40.189906 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c1300c67-b2af-4423-a62d-dcce0992fe27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:40.202929 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a5385-1de7-439b-8164-ff648741db34 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:40.268908 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995fe11-a3c0-4111-a90b-1e1341334f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:40.269625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995fe11-a3c0-4111-a90b-1e1341334f5f admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:40.270281 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 962/1938] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:13:40.270415 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995fe11-a3c0-4111-a90b-1e1341334f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:40.270415 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995fe11-a3c0-4111-a90b-1e1341334f5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:40.271091 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3995fe11-a3c0-4111-a90b-1e1341334f5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:40.272729 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3995fe11-a3c0-4111-a90b-1e1341334f5f admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:40.280060 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1300c67-b2af-4423-a62d-dcce0992fe27 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:40.281440 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c1300c67-b2af-4423-a62d-dcce0992fe27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:40.286136 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 977/1939] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:13:40.294601 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995fe11-a3c0-4111-a90b-1e1341334f5f admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:40.304762 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995fe11-a3c0-4111-a90b-1e1341334f5f admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:40.309458 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7bc6ab-4dce-4b4d-8044-a2fec09725d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:40.310111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bc6ab-4dce-4b4d-8044-a2fec09725d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:40.310397 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bc6ab-4dce-4b4d-8044-a2fec09725d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:40.310680 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bc6ab-4dce-4b4d-8044-a2fec09725d3 admin admin] PATH_INFO: `/v3/projects/95d4e8e5f33841b1b8fa3ec3c969d6d3/users/7b911e2bb4134ecd919be0287dd2db1f/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:40.326474 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7bc6ab-4dce-4b4d-8044-a2fec09725d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95d4e8e5f33841b1b8fa3ec3c969d6d3, user_id=7b911e2bb4134ecd919be0287dd2db1f, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:40.329348 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7bc6ab-4dce-4b4d-8044-a2fec09725d3 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:40.341653 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1300c67-b2af-4423-a62d-dcce0992fe27 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:40.345453 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b63db3c-0594-41c9-847c-762a5ccc55af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:40.346080 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63db3c-0594-41c9-847c-762a5ccc55af service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:40.346448 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63db3c-0594-41c9-847c-762a5ccc55af service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:40.346732 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63db3c-0594-41c9-847c-762a5ccc55af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:40.356492 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b63db3c-0594-41c9-847c-762a5ccc55af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:40.357930 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b63db3c-0594-41c9-847c-762a5ccc55af service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:40.372244 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 963/1940] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:13:40.398687 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 978/1941] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:40 2022] PUT /identity/v3/projects/95d4e8e5f33841b1b8fa3ec3c969d6d3/users/7b911e2bb4134ecd919be0287dd2db1f/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:40.411192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af96747-3edc-4ce4-882b-c54ed5d53509 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:40.411555 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af96747-3edc-4ce4-882b-c54ed5d53509 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:40.411925 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af96747-3edc-4ce4-882b-c54ed5d53509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:40.445415 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-1af96747-3edc-4ce4-882b-c54ed5d53509 None None] MFA Rules not processed for user `7b911e2bb4134ecd919be0287dd2db1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:40.446549 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1af96747-3edc-4ce4-882b-c54ed5d53509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:40.527963 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 964/1942] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:40 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:13:40.532571 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7bc6ab-4dce-4b4d-8044-a2fec09725d3 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:40.583826 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:40.603474 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7bc6ab-4dce-4b4d-8044-a2fec09725d3 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:40.611780 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843471e5-24c6-4afd-a4d5-0ceb99ea5c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:40.611780 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843471e5-24c6-4afd-a4d5-0ceb99ea5c10 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:40.611780 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843471e5-24c6-4afd-a4d5-0ceb99ea5c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:40.611780 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843471e5-24c6-4afd-a4d5-0ceb99ea5c10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:40.612795 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843471e5-24c6-4afd-a4d5-0ceb99ea5c10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:40.614668 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843471e5-24c6-4afd-a4d5-0ceb99ea5c10 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:40.762292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 979/1943] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:13:40 2022] POST /identity/v3/projects => generated 409 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:40.763548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af96747-3edc-4ce4-882b-c54ed5d53509 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:40.783295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af96747-3edc-4ce4-882b-c54ed5d53509 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:40.877081 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b21ee1-6f57-451d-ac1c-d0614d925936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:40.879749 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b21ee1-6f57-451d-ac1c-d0614d925936 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:40.880130 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b21ee1-6f57-451d-ac1c-d0614d925936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:40.880526 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b21ee1-6f57-451d-ac1c-d0614d925936 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:40.881572 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b21ee1-6f57-451d-ac1c-d0614d925936 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:40.883749 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b21ee1-6f57-451d-ac1c-d0614d925936 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:40.937863 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843471e5-24c6-4afd-a4d5-0ceb99ea5c10 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:40.944137 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-843471e5-24c6-4afd-a4d5-0ceb99ea5c10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:40.990880 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843471e5-24c6-4afd-a4d5-0ceb99ea5c10 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:40.997491 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb60205-64e7-475e-a169-9768a82cc204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:40.998493 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb60205-64e7-475e-a169-9768a82cc204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:40.999790 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb60205-64e7-475e-a169-9768a82cc204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:41.000313 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb60205-64e7-475e-a169-9768a82cc204 admin admin] PATH_INFO: `/v3/projects/25af358784094e94b201d1f7e49a4b3a` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:41.002041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb60205-64e7-475e-a169-9768a82cc204 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25af358784094e94b201d1f7e49a4b3a)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:41.004074 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 965/1944] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:13:40 2022] POST /identity/v3/users => generated 311 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:41.007742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb60205-64e7-475e-a169-9768a82cc204 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:41.007866 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b21ee1-6f57-451d-ac1c-d0614d925936 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:41.022116 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b21ee1-6f57-451d-ac1c-d0614d925936 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:41.027859 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027396ad-906d-472a-a9af-8b767850105c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:41.028972 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027396ad-906d-472a-a9af-8b767850105c admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:41.028972 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027396ad-906d-472a-a9af-8b767850105c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:41.029164 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027396ad-906d-472a-a9af-8b767850105c admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:41.029967 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027396ad-906d-472a-a9af-8b767850105c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:41.031968 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027396ad-906d-472a-a9af-8b767850105c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:41.047384 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 966/1945] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:13:41.052170 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027396ad-906d-472a-a9af-8b767850105c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:41.072504 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027396ad-906d-472a-a9af-8b767850105c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:41.083388 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71905db3-658e-48cd-8db4-f31f63455dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:41.084136 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71905db3-658e-48cd-8db4-f31f63455dd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:41.084418 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71905db3-658e-48cd-8db4-f31f63455dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:41.084730 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71905db3-658e-48cd-8db4-f31f63455dd6 admin admin] PATH_INFO: `/v3/projects/e5a8e718f5dd40c293483a65c8f666d6/users/dbe0ca1a1e544bc0a64d65b08dd96479/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:41.104920 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71905db3-658e-48cd-8db4-f31f63455dd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5a8e718f5dd40c293483a65c8f666d6, user_id=dbe0ca1a1e544bc0a64d65b08dd96479, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:41.109329 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71905db3-658e-48cd-8db4-f31f63455dd6 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:41.211282 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 967/1946] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:41 2022] PUT /identity/v3/projects/e5a8e718f5dd40c293483a65c8f666d6/users/dbe0ca1a1e544bc0a64d65b08dd96479/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:41.215218 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71905db3-658e-48cd-8db4-f31f63455dd6 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:41.218614 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-ffb60205-64e7-475e-a169-9768a82cc204 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25af358784094e94b201d1f7e49a4b3a', 'request_id': 'req-ffb60205-64e7-475e-a169-9768a82cc204'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:41.227220 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-ffb60205-64e7-475e-a169-9768a82cc204 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25af358784094e94b201d1f7e49a4b3a', 'request_id': 'req-ffb60205-64e7-475e-a169-9768a82cc204'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:41.285483 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71905db3-658e-48cd-8db4-f31f63455dd6 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:41.307767 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-ffb60205-64e7-475e-a169-9768a82cc204 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25af358784094e94b201d1f7e49a4b3a', 'request_id': 'req-ffb60205-64e7-475e-a169-9768a82cc204'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:41.321501 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-ffb60205-64e7-475e-a169-9768a82cc204 admin admin] The token cache is being invalidate because project 25af358784094e94b201d1f7e49a4b3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:13:41.321875 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-ffb60205-64e7-475e-a169-9768a82cc204 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:41.322570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-ffb60205-64e7-475e-a169-9768a82cc204 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:41.324475 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 980/1947] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:13:40 2022] DELETE /identity/v3/projects/25af358784094e94b201d1f7e49a4b3a => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:41.328606 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb60205-64e7-475e-a169-9768a82cc204 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:41.330030 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-ffb60205-64e7-475e-a169-9768a82cc204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:41.354513 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838c488-5942-48ad-aba4-cd32a5fceabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:41.355217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838c488-5942-48ad-aba4-cd32a5fceabd admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:41.355511 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838c488-5942-48ad-aba4-cd32a5fceabd admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:41.355780 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838c488-5942-48ad-aba4-cd32a5fceabd admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:41.356497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7838c488-5942-48ad-aba4-cd32a5fceabd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:41.358167 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7838c488-5942-48ad-aba4-cd32a5fceabd admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:41.369915 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 968/1948] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:13:41.374564 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838c488-5942-48ad-aba4-cd32a5fceabd admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:41.377208 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7838c488-5942-48ad-aba4-cd32a5fceabd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:41.415781 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb60205-64e7-475e-a169-9768a82cc204 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:41.417890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838c488-5942-48ad-aba4-cd32a5fceabd admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:41.421422 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3c56d-5f98-4a2b-80a3-e4f9d6d4c62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:41.421994 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3c56d-5f98-4a2b-80a3-e4f9d6d4c62c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:41.422366 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3c56d-5f98-4a2b-80a3-e4f9d6d4c62c admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:41.422700 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3c56d-5f98-4a2b-80a3-e4f9d6d4c62c admin admin] PATH_INFO: `/v3/users/13ffc8bb62014c3094e0b71f9a57ba3d` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:41.424227 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3c56d-5f98-4a2b-80a3-e4f9d6d4c62c admin admin] RBAC: Authorizing `identity:delete_user(user_id=13ffc8bb62014c3094e0b71f9a57ba3d)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:41.426957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3c56d-5f98-4a2b-80a3-e4f9d6d4c62c admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:41.429055 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914a9c3-8f6d-40fe-b0b8-3b6f25e3b9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:41.429751 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914a9c3-8f6d-40fe-b0b8-3b6f25e3b9cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:41.430033 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914a9c3-8f6d-40fe-b0b8-3b6f25e3b9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:41.430355 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914a9c3-8f6d-40fe-b0b8-3b6f25e3b9cd admin admin] PATH_INFO: `/v3/projects/e5a8e718f5dd40c293483a65c8f666d6/users/dbe0ca1a1e544bc0a64d65b08dd96479/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:41.432571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6914a9c3-8f6d-40fe-b0b8-3b6f25e3b9cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5a8e718f5dd40c293483a65c8f666d6, user_id=dbe0ca1a1e544bc0a64d65b08dd96479, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:41.435457 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6914a9c3-8f6d-40fe-b0b8-3b6f25e3b9cd admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:41.447081 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 969/1949] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:41 2022] PUT /identity/v3/projects/e5a8e718f5dd40c293483a65c8f666d6/users/dbe0ca1a1e544bc0a64d65b08dd96479/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:41.458168 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da4dfe-6cd4-494e-b5e0-9d4bb2da4113 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:41.458494 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da4dfe-6cd4-494e-b5e0-9d4bb2da4113 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:41.458773 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da4dfe-6cd4-494e-b5e0-9d4bb2da4113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:41.490022 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-78da4dfe-6cd4-494e-b5e0-9d4bb2da4113 None None] MFA Rules not processed for user `dbe0ca1a1e544bc0a64d65b08dd96479`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:41.491241 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-78da4dfe-6cd4-494e-b5e0-9d4bb2da4113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:41.576322 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 970/1950] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:41 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:13:41.582177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:41.601704 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-d0a3c56d-5f98-4a2b-80a3-e4f9d6d4c62c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13ffc8bb62014c3094e0b71f9a57ba3d'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:41.618500 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78da4dfe-6cd4-494e-b5e0-9d4bb2da4113 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:41.619955 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-78da4dfe-6cd4-494e-b5e0-9d4bb2da4113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:41.685616 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78da4dfe-6cd4-494e-b5e0-9d4bb2da4113 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:41.689409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b98cd-45a1-4266-a6d5-2bbb12301c90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:41.690579 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b98cd-45a1-4266-a6d5-2bbb12301c90 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:41.690883 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b98cd-45a1-4266-a6d5-2bbb12301c90 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:41.691164 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b98cd-45a1-4266-a6d5-2bbb12301c90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:41.695420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-d0a3c56d-5f98-4a2b-80a3-e4f9d6d4c62c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13ffc8bb62014c3094e0b71f9a57ba3d'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:41.696584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-d0a3c56d-5f98-4a2b-80a3-e4f9d6d4c62c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13ffc8bb62014c3094e0b71f9a57ba3d'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:41.702073 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77b98cd-45a1-4266-a6d5-2bbb12301c90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:41.705218 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77b98cd-45a1-4266-a6d5-2bbb12301c90 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:41.714574 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-d0a3c56d-5f98-4a2b-80a3-e4f9d6d4c62c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13ffc8bb62014c3094e0b71f9a57ba3d'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:41.715388 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-d0a3c56d-5f98-4a2b-80a3-e4f9d6d4c62c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13ffc8bb62014c3094e0b71f9a57ba3d'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:41.717975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 971/1951] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:41 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:13:41.739751 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 981/1952] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:13:41 2022] DELETE /identity/v3/users/13ffc8bb62014c3094e0b71f9a57ba3d => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:13:42.204974 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b98cd-45a1-4266-a6d5-2bbb12301c90 service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:42.216097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b98cd-45a1-4266-a6d5-2bbb12301c90 service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:42.292771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18cb978-9fa2-4919-8365-ea9ce3b5d891 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:42.293335 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18cb978-9fa2-4919-8365-ea9ce3b5d891 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:42.293690 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18cb978-9fa2-4919-8365-ea9ce3b5d891 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:42.294182 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18cb978-9fa2-4919-8365-ea9ce3b5d891 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:42.295986 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c18cb978-9fa2-4919-8365-ea9ce3b5d891 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:42.395354 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18cb978-9fa2-4919-8365-ea9ce3b5d891 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:42.397681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18cb978-9fa2-4919-8365-ea9ce3b5d891 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:42.411029 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 972/1953] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:42 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:42.492948 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3c56d-5f98-4a2b-80a3-e4f9d6d4c62c admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:42.494032 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-d0a3c56d-5f98-4a2b-80a3-e4f9d6d4c62c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:42.631581 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3c56d-5f98-4a2b-80a3-e4f9d6d4c62c admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:42.644762 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d0120-8a66-4841-af1e-6fde3717b69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:42.645742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d0120-8a66-4841-af1e-6fde3717b69a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:42.648605 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d0120-8a66-4841-af1e-6fde3717b69a admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:42.649127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d0120-8a66-4841-af1e-6fde3717b69a admin admin] PATH_INFO: `/v3/projects/63f33ed61f004c6cb170c85a56f6cdeb` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:42.650655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d0120-8a66-4841-af1e-6fde3717b69a admin admin] RBAC: Authorizing `identity:delete_project(project_id=63f33ed61f004c6cb170c85a56f6cdeb)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:42.653118 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d0120-8a66-4841-af1e-6fde3717b69a admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:42.838219 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-7b9d0120-8a66-4841-af1e-6fde3717b69a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63f33ed61f004c6cb170c85a56f6cdeb', 'request_id': 'req-7b9d0120-8a66-4841-af1e-6fde3717b69a'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:42.848605 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-7b9d0120-8a66-4841-af1e-6fde3717b69a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63f33ed61f004c6cb170c85a56f6cdeb', 'request_id': 'req-7b9d0120-8a66-4841-af1e-6fde3717b69a'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:42.957821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-7b9d0120-8a66-4841-af1e-6fde3717b69a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63f33ed61f004c6cb170c85a56f6cdeb', 'request_id': 'req-7b9d0120-8a66-4841-af1e-6fde3717b69a'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:42.976940 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-7b9d0120-8a66-4841-af1e-6fde3717b69a admin admin] The token cache is being invalidate because project 63f33ed61f004c6cb170c85a56f6cdeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:13:42.977238 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-7b9d0120-8a66-4841-af1e-6fde3717b69a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:42.978128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-7b9d0120-8a66-4841-af1e-6fde3717b69a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:42.980061 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 982/1954] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:13:42 2022] DELETE /identity/v3/projects/63f33ed61f004c6cb170c85a56f6cdeb => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:42.985896 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5cfcd-a0d7-4f12-aeff-c11d94c5cf07 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:42.986443 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5cfcd-a0d7-4f12-aeff-c11d94c5cf07 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:42.986560 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5cfcd-a0d7-4f12-aeff-c11d94c5cf07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:43.011136 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-8ac5cfcd-a0d7-4f12-aeff-c11d94c5cf07 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:43.012941 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8ac5cfcd-a0d7-4f12-aeff-c11d94c5cf07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:43.122372 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 973/1955] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:43.130686 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21b17d-1803-4a48-9b77-85d98fcf5dd1 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:43.131716 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21b17d-1803-4a48-9b77-85d98fcf5dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:43.131716 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21b17d-1803-4a48-9b77-85d98fcf5dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:43.171934 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-3c21b17d-1803-4a48-9b77-85d98fcf5dd1 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:43.175506 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-3c21b17d-1803-4a48-9b77-85d98fcf5dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:43.187595 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 983/1956] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:43.192786 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac5cfcd-a0d7-4f12-aeff-c11d94c5cf07 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:43.207974 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac5cfcd-a0d7-4f12-aeff-c11d94c5cf07 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:43.213925 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877bc797-e2bc-4461-8e23-629f35bf185d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:43.214742 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877bc797-e2bc-4461-8e23-629f35bf185d admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:43.215363 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877bc797-e2bc-4461-8e23-629f35bf185d admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:43.216434 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877bc797-e2bc-4461-8e23-629f35bf185d admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:43.217153 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877bc797-e2bc-4461-8e23-629f35bf185d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:13:43.217465 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877bc797-e2bc-4461-8e23-629f35bf185d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:43.219218 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877bc797-e2bc-4461-8e23-629f35bf185d admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:43.232852 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 974/1957] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:13:43.237592 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21b17d-1803-4a48-9b77-85d98fcf5dd1 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:43.273772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21b17d-1803-4a48-9b77-85d98fcf5dd1 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:43.285133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a903e4ec-f0db-496f-8f00-85184ef003e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:43.285881 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a903e4ec-f0db-496f-8f00-85184ef003e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:43.286760 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a903e4ec-f0db-496f-8f00-85184ef003e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:43.291274 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a903e4ec-f0db-496f-8f00-85184ef003e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:43.292390 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a903e4ec-f0db-496f-8f00-85184ef003e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:43.305998 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a903e4ec-f0db-496f-8f00-85184ef003e6 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:43.475370 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 984/1958] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:13:43 2022] POST /identity/v3/projects => generated 381 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:43.479553 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877bc797-e2bc-4461-8e23-629f35bf185d admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:43.494142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877bc797-e2bc-4461-8e23-629f35bf185d admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:43.497127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:43.571344 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51750f52-3c63-4d46-9780-6b422ce60af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:43.571344 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51750f52-3c63-4d46-9780-6b422ce60af4 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:43.571344 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51750f52-3c63-4d46-9780-6b422ce60af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:43.571344 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51750f52-3c63-4d46-9780-6b422ce60af4 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:43.571998 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51750f52-3c63-4d46-9780-6b422ce60af4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:43.571998 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51750f52-3c63-4d46-9780-6b422ce60af4 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:43.919886 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 975/1959] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:13:43 2022] POST /identity/v3/users => generated 291 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:13:43.923867 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a903e4ec-f0db-496f-8f00-85184ef003e6 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:43.937209 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a903e4ec-f0db-496f-8f00-85184ef003e6 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:43.946628 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33479702-516e-4080-a159-cdd6aa6778f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:43.947328 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33479702-516e-4080-a159-cdd6aa6778f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:43.947656 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33479702-516e-4080-a159-cdd6aa6778f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:43.948316 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33479702-516e-4080-a159-cdd6aa6778f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:43.950799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33479702-516e-4080-a159-cdd6aa6778f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:43.952581 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33479702-516e-4080-a159-cdd6aa6778f2 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:43.968040 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 985/1960] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:43.973686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51750f52-3c63-4d46-9780-6b422ce60af4 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:43.985491 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51750f52-3c63-4d46-9780-6b422ce60af4 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:43.991810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22400a77-842a-44c8-93d7-dcb3daf5dcbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:43.993162 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22400a77-842a-44c8-93d7-dcb3daf5dcbd admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:43.993456 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22400a77-842a-44c8-93d7-dcb3daf5dcbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:43.993736 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22400a77-842a-44c8-93d7-dcb3daf5dcbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:43.994447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22400a77-842a-44c8-93d7-dcb3daf5dcbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:43.996172 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22400a77-842a-44c8-93d7-dcb3daf5dcbd admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:44.015910 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 976/1961] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:44.017590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33479702-516e-4080-a159-cdd6aa6778f2 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:44.031676 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33479702-516e-4080-a159-cdd6aa6778f2 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:44.037071 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d46b92-d173-4a1f-882f-67b483a26a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:44.037766 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d46b92-d173-4a1f-882f-67b483a26a47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:44.038051 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d46b92-d173-4a1f-882f-67b483a26a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:44.038380 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d46b92-d173-4a1f-882f-67b483a26a47 admin admin] PATH_INFO: `/v3/projects/22b649660d4c4f91bb6cfc5ebdc2a3e3/users/e94f699226594dd6bd4bb708f7dd0781/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:44.060338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d46b92-d173-4a1f-882f-67b483a26a47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22b649660d4c4f91bb6cfc5ebdc2a3e3, user_id=e94f699226594dd6bd4bb708f7dd0781, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:44.063361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d46b92-d173-4a1f-882f-67b483a26a47 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:44.163780 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 986/1962] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:44 2022] PUT /identity/v3/projects/22b649660d4c4f91bb6cfc5ebdc2a3e3/users/e94f699226594dd6bd4bb708f7dd0781/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:44.176060 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7801c27-5f92-42fd-9113-b4fbdda38d01 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:44.176741 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7801c27-5f92-42fd-9113-b4fbdda38d01 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:44.177090 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7801c27-5f92-42fd-9113-b4fbdda38d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:44.214428 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-e7801c27-5f92-42fd-9113-b4fbdda38d01 None None] MFA Rules not processed for user `e94f699226594dd6bd4bb708f7dd0781`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:44.214428 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e7801c27-5f92-42fd-9113-b4fbdda38d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:44.298955 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 977/1963] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:13:44.303741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d46b92-d173-4a1f-882f-67b483a26a47 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:44.387939 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d46b92-d173-4a1f-882f-67b483a26a47 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:44.395544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a7162-5b2f-4e2e-bf57-f211ef6467a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:44.396190 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a7162-5b2f-4e2e-bf57-f211ef6467a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:44.396475 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a7162-5b2f-4e2e-bf57-f211ef6467a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:44.397121 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a7162-5b2f-4e2e-bf57-f211ef6467a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:44.397821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574a7162-5b2f-4e2e-bf57-f211ef6467a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:44.399853 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574a7162-5b2f-4e2e-bf57-f211ef6467a1 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:44.421372 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:44.486602 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7801c27-5f92-42fd-9113-b4fbdda38d01 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:44.491163 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 987/1964] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:13:44 2022] POST /identity/v3/projects => generated 379 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:44.514488 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7801c27-5f92-42fd-9113-b4fbdda38d01 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:44.630982 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a387d54-16fc-493a-b18c-37188938fe14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:44.632115 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a387d54-16fc-493a-b18c-37188938fe14 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:44.632899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a387d54-16fc-493a-b18c-37188938fe14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:44.633724 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a387d54-16fc-493a-b18c-37188938fe14 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:44.634419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a387d54-16fc-493a-b18c-37188938fe14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:44.642102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a387d54-16fc-493a-b18c-37188938fe14 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:44.749816 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 978/1965] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:13:44 2022] POST /identity/v3/users => generated 296 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:44.751512 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a7162-5b2f-4e2e-bf57-f211ef6467a1 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:44.764142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a7162-5b2f-4e2e-bf57-f211ef6467a1 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:44.774815 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85648404-b4cc-49ce-b0b8-fd030437d980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:44.774815 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85648404-b4cc-49ce-b0b8-fd030437d980 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:44.774815 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85648404-b4cc-49ce-b0b8-fd030437d980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:44.774815 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85648404-b4cc-49ce-b0b8-fd030437d980 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:44.775186 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85648404-b4cc-49ce-b0b8-fd030437d980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:44.775186 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85648404-b4cc-49ce-b0b8-fd030437d980 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:44.789383 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 988/1966] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:44.793519 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a387d54-16fc-493a-b18c-37188938fe14 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:44.808503 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a387d54-16fc-493a-b18c-37188938fe14 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:44.813510 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29138517-d681-4b03-a9f1-d7fd38bc07c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:44.814383 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29138517-d681-4b03-a9f1-d7fd38bc07c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:44.814513 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29138517-d681-4b03-a9f1-d7fd38bc07c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:44.815369 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29138517-d681-4b03-a9f1-d7fd38bc07c2 admin admin] PATH_INFO: `/v3/projects/7dba7d4cb640482a890855645053d4cf/users/c94a85102a204a728df7d32180e49771/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:44.855219 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29138517-d681-4b03-a9f1-d7fd38bc07c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dba7d4cb640482a890855645053d4cf, user_id=c94a85102a204a728df7d32180e49771, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:44.859616 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29138517-d681-4b03-a9f1-d7fd38bc07c2 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:44.952380 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 979/1967] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:44 2022] PUT /identity/v3/projects/7dba7d4cb640482a890855645053d4cf/users/c94a85102a204a728df7d32180e49771/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:44.954391 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85648404-b4cc-49ce-b0b8-fd030437d980 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:44.973453 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85648404-b4cc-49ce-b0b8-fd030437d980 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:45.053529 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42c6ec-2d97-4c85-9f98-5a402b51c950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:45.059955 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42c6ec-2d97-4c85-9f98-5a402b51c950 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:45.059955 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42c6ec-2d97-4c85-9f98-5a402b51c950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:45.059955 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42c6ec-2d97-4c85-9f98-5a402b51c950 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:45.059955 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a42c6ec-2d97-4c85-9f98-5a402b51c950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:45.059955 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a42c6ec-2d97-4c85-9f98-5a402b51c950 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:45.071812 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 989/1968] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:45.076758 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29138517-d681-4b03-a9f1-d7fd38bc07c2 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:45.102161 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29138517-d681-4b03-a9f1-d7fd38bc07c2 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:45.114071 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec10f1b-9e86-4388-b202-8bc0f2f6a259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:45.114352 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec10f1b-9e86-4388-b202-8bc0f2f6a259 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:45.114638 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec10f1b-9e86-4388-b202-8bc0f2f6a259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:45.114963 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec10f1b-9e86-4388-b202-8bc0f2f6a259 admin admin] PATH_INFO: `/v3/projects/7dba7d4cb640482a890855645053d4cf/users/c94a85102a204a728df7d32180e49771/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:45.116952 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec10f1b-9e86-4388-b202-8bc0f2f6a259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dba7d4cb640482a890855645053d4cf, user_id=c94a85102a204a728df7d32180e49771, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:45.119756 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec10f1b-9e86-4388-b202-8bc0f2f6a259 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:45.139338 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 980/1969] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:45 2022] PUT /identity/v3/projects/7dba7d4cb640482a890855645053d4cf/users/c94a85102a204a728df7d32180e49771/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:45.151143 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410cbadb-66be-4963-8ce2-a0571603183f None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:45.151438 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410cbadb-66be-4963-8ce2-a0571603183f None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:45.151725 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410cbadb-66be-4963-8ce2-a0571603183f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:45.191966 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-410cbadb-66be-4963-8ce2-a0571603183f None None] MFA Rules not processed for user `c94a85102a204a728df7d32180e49771`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:45.193468 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-410cbadb-66be-4963-8ce2-a0571603183f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:45.277498 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 990/1970] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:45 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:45.293007 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec10f1b-9e86-4388-b202-8bc0f2f6a259 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:45.294814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0ec10f1b-9e86-4388-b202-8bc0f2f6a259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:45.370215 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec10f1b-9e86-4388-b202-8bc0f2f6a259 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:45.372813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c88239-6fcc-4cfa-9e85-2ba1ca7570ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:45.373932 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c88239-6fcc-4cfa-9e85-2ba1ca7570ea service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:45.374224 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c88239-6fcc-4cfa-9e85-2ba1ca7570ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:45.374643 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c88239-6fcc-4cfa-9e85-2ba1ca7570ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:45.387236 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c88239-6fcc-4cfa-9e85-2ba1ca7570ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:45.395205 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c88239-6fcc-4cfa-9e85-2ba1ca7570ea service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:45.401733 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 981/1971] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:45 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:13:45.873018 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410cbadb-66be-4963-8ce2-a0571603183f None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:45.880646 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-410cbadb-66be-4963-8ce2-a0571603183f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:45.943778 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410cbadb-66be-4963-8ce2-a0571603183f None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:45.948764 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87abff1-84cd-4daa-a3c9-4415dcea661a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:45.949546 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87abff1-84cd-4daa-a3c9-4415dcea661a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:45.949840 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87abff1-84cd-4daa-a3c9-4415dcea661a admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:45.950175 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87abff1-84cd-4daa-a3c9-4415dcea661a admin admin] PATH_INFO: `/v3/users/805c135b9c7a423a89785f1069f1caec` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:45.951424 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87abff1-84cd-4daa-a3c9-4415dcea661a admin admin] RBAC: Authorizing `identity:delete_user(user_id=805c135b9c7a423a89785f1069f1caec)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:45.953419 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87abff1-84cd-4daa-a3c9-4415dcea661a admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:46.118801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:46.181150 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-c87abff1-84cd-4daa-a3c9-4415dcea661a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '805c135b9c7a423a89785f1069f1caec'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:46.225478 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:46.254001 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-c87abff1-84cd-4daa-a3c9-4415dcea661a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '805c135b9c7a423a89785f1069f1caec'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:46.254725 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-c87abff1-84cd-4daa-a3c9-4415dcea661a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '805c135b9c7a423a89785f1069f1caec'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:46.269060 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-c87abff1-84cd-4daa-a3c9-4415dcea661a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '805c135b9c7a423a89785f1069f1caec'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:46.269786 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-c87abff1-84cd-4daa-a3c9-4415dcea661a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '805c135b9c7a423a89785f1069f1caec'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:46.285181 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 991/1972] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:13:45 2022] DELETE /identity/v3/users/805c135b9c7a423a89785f1069f1caec => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:46.287087 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c9756-92e3-499f-96f1-62c1d0488365 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:46.287479 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c9756-92e3-499f-96f1-62c1d0488365 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:46.287694 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c9756-92e3-499f-96f1-62c1d0488365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:46.306449 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-183c9756-92e3-499f-96f1-62c1d0488365 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:46.307680 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-183c9756-92e3-499f-96f1-62c1d0488365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:46.368020 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 982/1973] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:46.378107 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87abff1-84cd-4daa-a3c9-4415dcea661a admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:46.379525 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-c87abff1-84cd-4daa-a3c9-4415dcea661a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:46.437253 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87abff1-84cd-4daa-a3c9-4415dcea661a admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:46.441292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a90187-d846-46b0-a2f3-eb2b01159301 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:46.441930 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a90187-d846-46b0-a2f3-eb2b01159301 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:46.442214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a90187-d846-46b0-a2f3-eb2b01159301 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:46.442490 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a90187-d846-46b0-a2f3-eb2b01159301 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:46.451447 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a90187-d846-46b0-a2f3-eb2b01159301 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:46.452892 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a90187-d846-46b0-a2f3-eb2b01159301 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:46.462871 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 992/1974] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:46.535484 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c9756-92e3-499f-96f1-62c1d0488365 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:46.536910 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-183c9756-92e3-499f-96f1-62c1d0488365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:46.605308 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c9756-92e3-499f-96f1-62c1d0488365 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:46.610676 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975cac8-c052-4b7a-8618-1a4859367058 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94a85102a204a728df7d32180e49771', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dba7d4cb640482a890855645053d4cf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:46.611362 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975cac8-c052-4b7a-8618-1a4859367058 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:46.612203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975cac8-c052-4b7a-8618-1a4859367058 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:46.612487 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975cac8-c052-4b7a-8618-1a4859367058 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:46.613193 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4975cac8-c052-4b7a-8618-1a4859367058 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:13:46.613504 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4975cac8-c052-4b7a-8618-1a4859367058 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:46.615145 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4975cac8-c052-4b7a-8618-1a4859367058 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:46.627854 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 983/1975] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:13:46.632019 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a90187-d846-46b0-a2f3-eb2b01159301 service neutron] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:46.642727 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a90187-d846-46b0-a2f3-eb2b01159301 service neutron] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:46.647962 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45cc86f-b5d8-4d02-a473-8cbf2a7e0fc4 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94a85102a204a728df7d32180e49771', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dba7d4cb640482a890855645053d4cf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:46.648584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45cc86f-b5d8-4d02-a473-8cbf2a7e0fc4 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:46.648874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45cc86f-b5d8-4d02-a473-8cbf2a7e0fc4 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:46.649158 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45cc86f-b5d8-4d02-a473-8cbf2a7e0fc4 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:46.649868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45cc86f-b5d8-4d02-a473-8cbf2a7e0fc4 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:46.651839 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45cc86f-b5d8-4d02-a473-8cbf2a7e0fc4 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:46.743601 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 993/1976] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:13:46 2022] POST /identity/v3/projects => generated 355 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:46.746253 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975cac8-c052-4b7a-8618-1a4859367058 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:46.756562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975cac8-c052-4b7a-8618-1a4859367058 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:46.816967 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89d2e2-62c6-4e6d-ab89-5a4de3cd73b8 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94a85102a204a728df7d32180e49771', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dba7d4cb640482a890855645053d4cf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:46.817692 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89d2e2-62c6-4e6d-ab89-5a4de3cd73b8 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:46.817975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89d2e2-62c6-4e6d-ab89-5a4de3cd73b8 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:46.818256 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89d2e2-62c6-4e6d-ab89-5a4de3cd73b8 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:46.818991 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89d2e2-62c6-4e6d-ab89-5a4de3cd73b8 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:13:46.819306 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89d2e2-62c6-4e6d-ab89-5a4de3cd73b8 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:46.820918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89d2e2-62c6-4e6d-ab89-5a4de3cd73b8 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:46.833769 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 984/1977] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 05:13:46.871951 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45cc86f-b5d8-4d02-a473-8cbf2a7e0fc4 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:46.882443 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45cc86f-b5d8-4d02-a473-8cbf2a7e0fc4 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:46.886988 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455058b8-5588-46be-9aa0-e92d42a9ecb9 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94a85102a204a728df7d32180e49771', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dba7d4cb640482a890855645053d4cf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:46.893608 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455058b8-5588-46be-9aa0-e92d42a9ecb9 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:46.893608 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455058b8-5588-46be-9aa0-e92d42a9ecb9 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:46.893608 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455058b8-5588-46be-9aa0-e92d42a9ecb9 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] PATH_INFO: `/v3/projects/758cb434f5e945ae8995868b1f7299c3` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:46.893608 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455058b8-5588-46be-9aa0-e92d42a9ecb9 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] RBAC: Authorizing `identity:delete_project(project_id=758cb434f5e945ae8995868b1f7299c3)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:46.893969 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455058b8-5588-46be-9aa0-e92d42a9ecb9 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:46.919905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89d2e2-62c6-4e6d-ab89-5a4de3cd73b8 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:46.921820 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ed89d2e2-62c6-4e6d-ab89-5a4de3cd73b8 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:47.003042 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89d2e2-62c6-4e6d-ab89-5a4de3cd73b8 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:47.008225 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f4fe04-098a-4341-b7bb-8cff614f3c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:47.008881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f4fe04-098a-4341-b7bb-8cff614f3c5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:47.009173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f4fe04-098a-4341-b7bb-8cff614f3c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:47.009472 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f4fe04-098a-4341-b7bb-8cff614f3c5a admin admin] PATH_INFO: `/v3/projects/2cdc9c3e28ba491faf3616c06fcb3beb` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:47.010923 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f4fe04-098a-4341-b7bb-8cff614f3c5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cdc9c3e28ba491faf3616c06fcb3beb)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:47.012692 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f4fe04-098a-4341-b7bb-8cff614f3c5a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:47.023783 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-455058b8-5588-46be-9aa0-e92d42a9ecb9 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '758cb434f5e945ae8995868b1f7299c3', 'request_id': 'req-455058b8-5588-46be-9aa0-e92d42a9ecb9'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:47.030792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-455058b8-5588-46be-9aa0-e92d42a9ecb9 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '758cb434f5e945ae8995868b1f7299c3', 'request_id': 'req-455058b8-5588-46be-9aa0-e92d42a9ecb9'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:47.113497 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-455058b8-5588-46be-9aa0-e92d42a9ecb9 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '758cb434f5e945ae8995868b1f7299c3', 'request_id': 'req-455058b8-5588-46be-9aa0-e92d42a9ecb9'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:47.126582 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-455058b8-5588-46be-9aa0-e92d42a9ecb9 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] The token cache is being invalidate because project 758cb434f5e945ae8995868b1f7299c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:13:47.127423 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-455058b8-5588-46be-9aa0-e92d42a9ecb9 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:47.127693 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-455058b8-5588-46be-9aa0-e92d42a9ecb9 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:47.129247 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 994/1978] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:13:46 2022] DELETE /identity/v3/projects/758cb434f5e945ae8995868b1f7299c3 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:47.132330 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-35f4fe04-098a-4341-b7bb-8cff614f3c5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cdc9c3e28ba491faf3616c06fcb3beb', 'request_id': 'req-35f4fe04-098a-4341-b7bb-8cff614f3c5a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:47.203356 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-35f4fe04-098a-4341-b7bb-8cff614f3c5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cdc9c3e28ba491faf3616c06fcb3beb', 'request_id': 'req-35f4fe04-098a-4341-b7bb-8cff614f3c5a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:47.209694 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-35f4fe04-098a-4341-b7bb-8cff614f3c5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cdc9c3e28ba491faf3616c06fcb3beb', 'request_id': 'req-35f4fe04-098a-4341-b7bb-8cff614f3c5a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:47.221988 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-35f4fe04-098a-4341-b7bb-8cff614f3c5a admin admin] The token cache is being invalidate because project 2cdc9c3e28ba491faf3616c06fcb3beb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:13:47.222278 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-35f4fe04-098a-4341-b7bb-8cff614f3c5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:47.223298 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-35f4fe04-098a-4341-b7bb-8cff614f3c5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:47.225116 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 985/1979] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:13:46 2022] DELETE /identity/v3/projects/2cdc9c3e28ba491faf3616c06fcb3beb => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:47.229125 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455058b8-5588-46be-9aa0-e92d42a9ecb9 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:47.230602 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-455058b8-5588-46be-9aa0-e92d42a9ecb9 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:47.303718 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455058b8-5588-46be-9aa0-e92d42a9ecb9 tempest-VolumeQuotaClassesTest-450630067 tempest-VolumeQuotaClassesTest-450630067-project-admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:47.364300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd23b38-5f1e-400c-85be-a6bda01d9730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:47.365118 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd23b38-5f1e-400c-85be-a6bda01d9730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:47.365483 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd23b38-5f1e-400c-85be-a6bda01d9730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:47.365831 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd23b38-5f1e-400c-85be-a6bda01d9730 admin admin] PATH_INFO: `/v3/users/e562ea6fe897451a96db5475cb2e5794` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:47.367216 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd23b38-5f1e-400c-85be-a6bda01d9730 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e562ea6fe897451a96db5475cb2e5794)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:47.369288 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd23b38-5f1e-400c-85be-a6bda01d9730 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:47.586794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-fcd23b38-5f1e-400c-85be-a6bda01d9730 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e562ea6fe897451a96db5475cb2e5794'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:47.677624 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-fcd23b38-5f1e-400c-85be-a6bda01d9730 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e562ea6fe897451a96db5475cb2e5794'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:47.678261 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-fcd23b38-5f1e-400c-85be-a6bda01d9730 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e562ea6fe897451a96db5475cb2e5794'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:47.691870 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-fcd23b38-5f1e-400c-85be-a6bda01d9730 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e562ea6fe897451a96db5475cb2e5794'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:47.692561 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-fcd23b38-5f1e-400c-85be-a6bda01d9730 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e562ea6fe897451a96db5475cb2e5794'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:47.705143 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 995/1980] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:13:47 2022] DELETE /identity/v3/users/e562ea6fe897451a96db5475cb2e5794 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:47.755731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f4fe04-098a-4341-b7bb-8cff614f3c5a admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:47.757271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-35f4fe04-098a-4341-b7bb-8cff614f3c5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:47.770999 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f4fe04-098a-4341-b7bb-8cff614f3c5a admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:47.816843 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:47.829248 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9868c5-2126-4d7c-a758-16861d59222b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:47.829893 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9868c5-2126-4d7c-a758-16861d59222b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:47.830732 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9868c5-2126-4d7c-a758-16861d59222b admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:47.831053 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9868c5-2126-4d7c-a758-16861d59222b admin admin] PATH_INFO: `/v3/users/e94f699226594dd6bd4bb708f7dd0781` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:47.832226 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9868c5-2126-4d7c-a758-16861d59222b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e94f699226594dd6bd4bb708f7dd0781)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:47.834157 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9868c5-2126-4d7c-a758-16861d59222b admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:47.883759 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:48.006135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-3d9868c5-2126-4d7c-a758-16861d59222b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e94f699226594dd6bd4bb708f7dd0781'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:48.055470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-3d9868c5-2126-4d7c-a758-16861d59222b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e94f699226594dd6bd4bb708f7dd0781'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:48.056156 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-3d9868c5-2126-4d7c-a758-16861d59222b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e94f699226594dd6bd4bb708f7dd0781'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:48.068733 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-3d9868c5-2126-4d7c-a758-16861d59222b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e94f699226594dd6bd4bb708f7dd0781'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:48.069379 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-3d9868c5-2126-4d7c-a758-16861d59222b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e94f699226594dd6bd4bb708f7dd0781'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:48.090417 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 986/1981] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:13:47 2022] DELETE /identity/v3/users/e94f699226594dd6bd4bb708f7dd0781 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:48.090565 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd23b38-5f1e-400c-85be-a6bda01d9730 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:48.090713 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b483dc9-8848-4e5a-b44f-9b52cf320974 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:48.090713 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b483dc9-8848-4e5a-b44f-9b52cf320974 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:48.090874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-fcd23b38-5f1e-400c-85be-a6bda01d9730 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:48.090969 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b483dc9-8848-4e5a-b44f-9b52cf320974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:48.105623 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd23b38-5f1e-400c-85be-a6bda01d9730 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:48.118212 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-5b483dc9-8848-4e5a-b44f-9b52cf320974 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:48.118212 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-5b483dc9-8848-4e5a-b44f-9b52cf320974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:48.175302 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff9d3b-797a-4c17-b8f6-062c33b3ddfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:48.176048 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff9d3b-797a-4c17-b8f6-062c33b3ddfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:48.176222 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff9d3b-797a-4c17-b8f6-062c33b3ddfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:48.176444 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff9d3b-797a-4c17-b8f6-062c33b3ddfd admin admin] PATH_INFO: `/v3/projects/1135fd2f6560485da4aa3927a568b37c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:48.177643 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff9d3b-797a-4c17-b8f6-062c33b3ddfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1135fd2f6560485da4aa3927a568b37c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:48.183411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff9d3b-797a-4c17-b8f6-062c33b3ddfd admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:48.186160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 987/1982] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:48.196104 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b483dc9-8848-4e5a-b44f-9b52cf320974 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:48.197603 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-5b483dc9-8848-4e5a-b44f-9b52cf320974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:48.255928 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b483dc9-8848-4e5a-b44f-9b52cf320974 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:48.259704 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8cb700-bf84-4bce-a84a-3a914fadac36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:48.260330 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cb700-bf84-4bce-a84a-3a914fadac36 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:48.260616 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cb700-bf84-4bce-a84a-3a914fadac36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:48.260895 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cb700-bf84-4bce-a84a-3a914fadac36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:48.271074 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8cb700-bf84-4bce-a84a-3a914fadac36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:48.275266 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8cb700-bf84-4bce-a84a-3a914fadac36 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:48.282772 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 988/1983] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:13:48.334108 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-bdff9d3b-797a-4c17-b8f6-062c33b3ddfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1135fd2f6560485da4aa3927a568b37c', 'request_id': 'req-bdff9d3b-797a-4c17-b8f6-062c33b3ddfd'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:48.341062 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-bdff9d3b-797a-4c17-b8f6-062c33b3ddfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1135fd2f6560485da4aa3927a568b37c', 'request_id': 'req-bdff9d3b-797a-4c17-b8f6-062c33b3ddfd'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:48.409984 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-bdff9d3b-797a-4c17-b8f6-062c33b3ddfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1135fd2f6560485da4aa3927a568b37c', 'request_id': 'req-bdff9d3b-797a-4c17-b8f6-062c33b3ddfd'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:48.421171 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-bdff9d3b-797a-4c17-b8f6-062c33b3ddfd admin admin] The token cache is being invalidate because project 1135fd2f6560485da4aa3927a568b37c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:13:48.421465 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-bdff9d3b-797a-4c17-b8f6-062c33b3ddfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:48.422180 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-bdff9d3b-797a-4c17-b8f6-062c33b3ddfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:48.423926 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 996/1984] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:13:48 2022] DELETE /identity/v3/projects/1135fd2f6560485da4aa3927a568b37c => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:48.429966 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf0097-5f5b-4a93-80c9-83b87e7524c3 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:48.430200 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf0097-5f5b-4a93-80c9-83b87e7524c3 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:48.430484 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf0097-5f5b-4a93-80c9-83b87e7524c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:48.450368 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-f0cf0097-5f5b-4a93-80c9-83b87e7524c3 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:48.451610 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f0cf0097-5f5b-4a93-80c9-83b87e7524c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:48.523967 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 989/1985] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:48.526300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b870e-dc31-4385-8ca9-49185fa104b9 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:48.526691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b870e-dc31-4385-8ca9-49185fa104b9 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:48.527249 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b870e-dc31-4385-8ca9-49185fa104b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:48.553818 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-af7b870e-dc31-4385-8ca9-49185fa104b9 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:48.555219 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-af7b870e-dc31-4385-8ca9-49185fa104b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:48.564941 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 997/1986] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:48.569219 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf0097-5f5b-4a93-80c9-83b87e7524c3 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:48.579295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf0097-5f5b-4a93-80c9-83b87e7524c3 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:48.585191 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf2a4f1-f3df-4a5e-a4ec-746bf73ca2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:48.585839 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2a4f1-f3df-4a5e-a4ec-746bf73ca2ff admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:48.586129 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2a4f1-f3df-4a5e-a4ec-746bf73ca2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:48.586407 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2a4f1-f3df-4a5e-a4ec-746bf73ca2ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:48.587172 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf2a4f1-f3df-4a5e-a4ec-746bf73ca2ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:13:48.590321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf2a4f1-f3df-4a5e-a4ec-746bf73ca2ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:48.590321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf2a4f1-f3df-4a5e-a4ec-746bf73ca2ff admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:48.602528 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 990/1987] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 05:13:48.607074 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b870e-dc31-4385-8ca9-49185fa104b9 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:48.617637 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b870e-dc31-4385-8ca9-49185fa104b9 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:48.622545 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c31d5-516e-4d1a-b134-a349826bfa0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:48.623238 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c31d5-516e-4d1a-b134-a349826bfa0e admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:48.623529 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c31d5-516e-4d1a-b134-a349826bfa0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:48.623807 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c31d5-516e-4d1a-b134-a349826bfa0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:48.624485 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c31d5-516e-4d1a-b134-a349826bfa0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:48.626505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c31d5-516e-4d1a-b134-a349826bfa0e admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:48.728522 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 998/1988] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:13:48 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:48.729770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf2a4f1-f3df-4a5e-a4ec-746bf73ca2ff admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:48.742453 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf2a4f1-f3df-4a5e-a4ec-746bf73ca2ff admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:48.809968 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f47b8-88ef-4683-9a07-6153041eb162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:48.810678 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f47b8-88ef-4683-9a07-6153041eb162 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:48.810890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f47b8-88ef-4683-9a07-6153041eb162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:48.811171 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f47b8-88ef-4683-9a07-6153041eb162 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:48.812056 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f47b8-88ef-4683-9a07-6153041eb162 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:48.813997 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f47b8-88ef-4683-9a07-6153041eb162 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:48.933879 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 991/1989] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:13:48 2022] POST /identity/v3/users => generated 287 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:13:48.937981 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c31d5-516e-4d1a-b134-a349826bfa0e admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:48.951821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c31d5-516e-4d1a-b134-a349826bfa0e admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:48.956895 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e58135-59be-44aa-ad57-d7682c762492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:48.957584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e58135-59be-44aa-ad57-d7682c762492 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:48.957871 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e58135-59be-44aa-ad57-d7682c762492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:48.958154 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e58135-59be-44aa-ad57-d7682c762492 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:48.958873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e58135-59be-44aa-ad57-d7682c762492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:48.960505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e58135-59be-44aa-ad57-d7682c762492 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:48.971056 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 999/1990] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:48.975635 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f47b8-88ef-4683-9a07-6153041eb162 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:48.975842 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:48.986059 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f47b8-88ef-4683-9a07-6153041eb162 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:48.991268 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b133ecc-f4b4-455b-862b-7abcc80d03d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:48.991909 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b133ecc-f4b4-455b-862b-7abcc80d03d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:48.992192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b133ecc-f4b4-455b-862b-7abcc80d03d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:48.992617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b133ecc-f4b4-455b-862b-7abcc80d03d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:48.993347 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b133ecc-f4b4-455b-862b-7abcc80d03d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:48.994967 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b133ecc-f4b4-455b-862b-7abcc80d03d8 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:49.005462 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 992/1991] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:49.010099 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e58135-59be-44aa-ad57-d7682c762492 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:49.011993 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b133ecc-f4b4-455b-862b-7abcc80d03d8 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:49.013585 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7b133ecc-f4b4-455b-862b-7abcc80d03d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:49.022535 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e58135-59be-44aa-ad57-d7682c762492 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:49.026834 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:49.030821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a454d53-6ae2-4e2a-8b50-8b6a7e1abf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:49.030821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a454d53-6ae2-4e2a-8b50-8b6a7e1abf67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:49.030821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a454d53-6ae2-4e2a-8b50-8b6a7e1abf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:49.030821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a454d53-6ae2-4e2a-8b50-8b6a7e1abf67 admin admin] PATH_INFO: `/v3/projects/293662ace54645c8bbdfcdf3a80aa8d8/users/8e895dabe56e4269bc8197324c8f8ccb/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:49.036470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b133ecc-f4b4-455b-862b-7abcc80d03d8 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:49.041239 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0224f7-317e-4af5-8012-a8ae7e4dfa03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:49.041888 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0224f7-317e-4af5-8012-a8ae7e4dfa03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:49.042176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0224f7-317e-4af5-8012-a8ae7e4dfa03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:49.042452 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0224f7-317e-4af5-8012-a8ae7e4dfa03 admin admin] PATH_INFO: `/v3/projects/22b649660d4c4f91bb6cfc5ebdc2a3e3` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:49.043774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0224f7-317e-4af5-8012-a8ae7e4dfa03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22b649660d4c4f91bb6cfc5ebdc2a3e3)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:49.045837 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0224f7-317e-4af5-8012-a8ae7e4dfa03 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:49.048261 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a454d53-6ae2-4e2a-8b50-8b6a7e1abf67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=293662ace54645c8bbdfcdf3a80aa8d8, user_id=8e895dabe56e4269bc8197324c8f8ccb, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:49.051575 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a454d53-6ae2-4e2a-8b50-8b6a7e1abf67 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:49.118695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1000/1992] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:49 2022] PUT /identity/v3/projects/293662ace54645c8bbdfcdf3a80aa8d8/users/8e895dabe56e4269bc8197324c8f8ccb/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:13:49.130626 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9794c-f9aa-4ea0-bed0-81c443eea2c2 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:49.130957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9794c-f9aa-4ea0-bed0-81c443eea2c2 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:49.131243 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9794c-f9aa-4ea0-bed0-81c443eea2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:49.159165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-98c9794c-f9aa-4ea0-bed0-81c443eea2c2 None None] MFA Rules not processed for user `8e895dabe56e4269bc8197324c8f8ccb`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:49.160393 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-98c9794c-f9aa-4ea0-bed0-81c443eea2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:49.212492 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2f0224f7-317e-4af5-8012-a8ae7e4dfa03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22b649660d4c4f91bb6cfc5ebdc2a3e3', 'request_id': 'req-2f0224f7-317e-4af5-8012-a8ae7e4dfa03'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:49.240162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1001/1993] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:13:49.251230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9794c-f9aa-4ea0-bed0-81c443eea2c2 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:49.253095 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-98c9794c-f9aa-4ea0-bed0-81c443eea2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:49.287402 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2f0224f7-317e-4af5-8012-a8ae7e4dfa03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22b649660d4c4f91bb6cfc5ebdc2a3e3', 'request_id': 'req-2f0224f7-317e-4af5-8012-a8ae7e4dfa03'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:49.293989 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2f0224f7-317e-4af5-8012-a8ae7e4dfa03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22b649660d4c4f91bb6cfc5ebdc2a3e3', 'request_id': 'req-2f0224f7-317e-4af5-8012-a8ae7e4dfa03'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:49.306256 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2f0224f7-317e-4af5-8012-a8ae7e4dfa03 admin admin] The token cache is being invalidate because project 22b649660d4c4f91bb6cfc5ebdc2a3e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:13:49.306556 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2f0224f7-317e-4af5-8012-a8ae7e4dfa03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:49.307265 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2f0224f7-317e-4af5-8012-a8ae7e4dfa03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:49.309019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 993/1994] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:13:49 2022] DELETE /identity/v3/projects/22b649660d4c4f91bb6cfc5ebdc2a3e3 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:13:49.309685 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9794c-f9aa-4ea0-bed0-81c443eea2c2 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:49.315017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0224f7-317e-4af5-8012-a8ae7e4dfa03 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:49.315017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2f0224f7-317e-4af5-8012-a8ae7e4dfa03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:49.374041 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958fcd7-edd1-4ca4-9a7c-1037db5775d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:49.374949 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958fcd7-edd1-4ca4-9a7c-1037db5775d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:49.375582 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958fcd7-edd1-4ca4-9a7c-1037db5775d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:49.376070 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958fcd7-edd1-4ca4-9a7c-1037db5775d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:49.379007 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-e958fcd7-edd1-4ca4-9a7c-1037db5775d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:49.386052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0224f7-317e-4af5-8012-a8ae7e4dfa03 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:49.399284 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e88c829-4c3e-46f9-a4e0-fa6d33e78b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:49.400049 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88c829-4c3e-46f9-a4e0-fa6d33e78b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:49.400416 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88c829-4c3e-46f9-a4e0-fa6d33e78b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:49.400798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88c829-4c3e-46f9-a4e0-fa6d33e78b61 admin admin] PATH_INFO: `/v3/users/c94a85102a204a728df7d32180e49771` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:49.402472 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e88c829-4c3e-46f9-a4e0-fa6d33e78b61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c94a85102a204a728df7d32180e49771)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:49.405167 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e88c829-4c3e-46f9-a4e0-fa6d33e78b61 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:49.457987 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e958fcd7-edd1-4ca4-9a7c-1037db5775d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:49.459472 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e958fcd7-edd1-4ca4-9a7c-1037db5775d6 service cinder] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:49.469013 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1002/1995] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:13:49.492111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958fcd7-edd1-4ca4-9a7c-1037db5775d6 service cinder] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:49.493586 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-e958fcd7-edd1-4ca4-9a7c-1037db5775d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:49.509565 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958fcd7-edd1-4ca4-9a7c-1037db5775d6 service cinder] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:49.514371 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f564c-5aaa-41b7-8725-e5eddbad3e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:49.515057 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f564c-5aaa-41b7-8725-e5eddbad3e30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:49.515341 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f564c-5aaa-41b7-8725-e5eddbad3e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:49.515796 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f564c-5aaa-41b7-8725-e5eddbad3e30 admin admin] PATH_INFO: `/v3/users/8e895dabe56e4269bc8197324c8f8ccb` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:49.517073 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f564c-5aaa-41b7-8725-e5eddbad3e30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e895dabe56e4269bc8197324c8f8ccb)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:49.519622 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f564c-5aaa-41b7-8725-e5eddbad3e30 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:49.610859 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7e88c829-4c3e-46f9-a4e0-fa6d33e78b61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c94a85102a204a728df7d32180e49771'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:49.712155 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7e88c829-4c3e-46f9-a4e0-fa6d33e78b61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c94a85102a204a728df7d32180e49771'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:49.712797 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7e88c829-4c3e-46f9-a4e0-fa6d33e78b61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c94a85102a204a728df7d32180e49771'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:49.728052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-b42f564c-5aaa-41b7-8725-e5eddbad3e30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e895dabe56e4269bc8197324c8f8ccb'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:49.729994 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7e88c829-4c3e-46f9-a4e0-fa6d33e78b61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c94a85102a204a728df7d32180e49771'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:49.730721 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7e88c829-4c3e-46f9-a4e0-fa6d33e78b61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c94a85102a204a728df7d32180e49771'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:49.744604 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 994/1996] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:13:49 2022] DELETE /identity/v3/users/c94a85102a204a728df7d32180e49771 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:49.815582 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-b42f564c-5aaa-41b7-8725-e5eddbad3e30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e895dabe56e4269bc8197324c8f8ccb'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:49.816608 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-b42f564c-5aaa-41b7-8725-e5eddbad3e30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e895dabe56e4269bc8197324c8f8ccb'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:49.830961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-b42f564c-5aaa-41b7-8725-e5eddbad3e30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e895dabe56e4269bc8197324c8f8ccb'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:49.831679 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-b42f564c-5aaa-41b7-8725-e5eddbad3e30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e895dabe56e4269bc8197324c8f8ccb'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:49.844638 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1003/1997] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:13:49 2022] DELETE /identity/v3/users/8e895dabe56e4269bc8197324c8f8ccb => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:13:49.849634 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22913f93-ad6c-4458-b30f-f941f1309583 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:49.852565 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22913f93-ad6c-4458-b30f-f941f1309583 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:49.852565 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22913f93-ad6c-4458-b30f-f941f1309583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:49.869838 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-22913f93-ad6c-4458-b30f-f941f1309583 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:49.871120 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-22913f93-ad6c-4458-b30f-f941f1309583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:49.913973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:49.933529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 995/1998] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:49.944814 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f564c-5aaa-41b7-8725-e5eddbad3e30 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:49.947382 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-b42f564c-5aaa-41b7-8725-e5eddbad3e30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:50.015981 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f564c-5aaa-41b7-8725-e5eddbad3e30 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:50.020221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a10ae28-7fae-4a5d-b35a-037d7eb21340 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:50.020983 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10ae28-7fae-4a5d-b35a-037d7eb21340 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:50.022607 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10ae28-7fae-4a5d-b35a-037d7eb21340 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:50.022607 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10ae28-7fae-4a5d-b35a-037d7eb21340 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:50.040589 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a10ae28-7fae-4a5d-b35a-037d7eb21340 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:50.042489 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a10ae28-7fae-4a5d-b35a-037d7eb21340 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:50.071726 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1004/1999] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:50.204450 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:50.292218 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22913f93-ad6c-4458-b30f-f941f1309583 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:50.293934 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-22913f93-ad6c-4458-b30f-f941f1309583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:50.319538 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22913f93-ad6c-4458-b30f-f941f1309583 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:50.325057 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfb486b-5fe8-47e6-89cc-1bf1e30b4f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:50.325862 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfb486b-5fe8-47e6-89cc-1bf1e30b4f76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:50.326243 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfb486b-5fe8-47e6-89cc-1bf1e30b4f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:50.327210 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfb486b-5fe8-47e6-89cc-1bf1e30b4f76 admin admin] PATH_INFO: `/v3/projects/7dba7d4cb640482a890855645053d4cf` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:50.328545 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfb486b-5fe8-47e6-89cc-1bf1e30b4f76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dba7d4cb640482a890855645053d4cf)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:50.331203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfb486b-5fe8-47e6-89cc-1bf1e30b4f76 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:50.477613 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7bfb486b-5fe8-47e6-89cc-1bf1e30b4f76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dba7d4cb640482a890855645053d4cf', 'request_id': 'req-7bfb486b-5fe8-47e6-89cc-1bf1e30b4f76'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:50.548076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a10ae28-7fae-4a5d-b35a-037d7eb21340 service neutron] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:50.548076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-7a10ae28-7fae-4a5d-b35a-037d7eb21340 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:50.560981 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a10ae28-7fae-4a5d-b35a-037d7eb21340 service neutron] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:50.576542 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7bfb486b-5fe8-47e6-89cc-1bf1e30b4f76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dba7d4cb640482a890855645053d4cf', 'request_id': 'req-7bfb486b-5fe8-47e6-89cc-1bf1e30b4f76'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:50.583626 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7bfb486b-5fe8-47e6-89cc-1bf1e30b4f76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dba7d4cb640482a890855645053d4cf', 'request_id': 'req-7bfb486b-5fe8-47e6-89cc-1bf1e30b4f76'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:50.595860 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7bfb486b-5fe8-47e6-89cc-1bf1e30b4f76 admin admin] The token cache is being invalidate because project 7dba7d4cb640482a890855645053d4cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:13:50.596239 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7bfb486b-5fe8-47e6-89cc-1bf1e30b4f76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:50.596973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7bfb486b-5fe8-47e6-89cc-1bf1e30b4f76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:50.598774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 996/2000] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:13:50 2022] DELETE /identity/v3/projects/7dba7d4cb640482a890855645053d4cf => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:50.604518 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b00d5-868c-46e7-a27d-847481cc148e None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:50.604815 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b00d5-868c-46e7-a27d-847481cc148e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:50.605099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b00d5-868c-46e7-a27d-847481cc148e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:50.622372 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06b79c-00da-486f-8c70-45207ed1678d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:50.628498 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06b79c-00da-486f-8c70-45207ed1678d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:50.628498 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06b79c-00da-486f-8c70-45207ed1678d admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:50.628498 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06b79c-00da-486f-8c70-45207ed1678d admin admin] PATH_INFO: `/v3/projects/293662ace54645c8bbdfcdf3a80aa8d8` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:50.628498 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f06b79c-00da-486f-8c70-45207ed1678d admin admin] RBAC: Authorizing `identity:delete_project(project_id=293662ace54645c8bbdfcdf3a80aa8d8)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:50.628949 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-b90b00d5-868c-46e7-a27d-847481cc148e None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:50.628949 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b90b00d5-868c-46e7-a27d-847481cc148e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:50.629122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f06b79c-00da-486f-8c70-45207ed1678d admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:50.637545 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 997/2001] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:50.646258 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60a8f3-8b71-4457-84a3-f97195dfe812 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:50.651489 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60a8f3-8b71-4457-84a3-f97195dfe812 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:50.651489 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60a8f3-8b71-4457-84a3-f97195dfe812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:50.651489 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:50.670016 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-be60a8f3-8b71-4457-84a3-f97195dfe812 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:50.671388 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-be60a8f3-8b71-4457-84a3-f97195dfe812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:50.680450 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 998/2002] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:50.684645 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60a8f3-8b71-4457-84a3-f97195dfe812 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:50.697412 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60a8f3-8b71-4457-84a3-f97195dfe812 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:50.698739 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4eeea-d231-405e-8dfb-c3ffe05d97e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:50.699413 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4eeea-d231-405e-8dfb-c3ffe05d97e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:50.699710 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4eeea-d231-405e-8dfb-c3ffe05d97e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:50.699972 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4eeea-d231-405e-8dfb-c3ffe05d97e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:50.700694 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb4eeea-d231-405e-8dfb-c3ffe05d97e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:13:50.701006 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb4eeea-d231-405e-8dfb-c3ffe05d97e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:50.702608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb4eeea-d231-405e-8dfb-c3ffe05d97e7 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:50.714472 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 999/2003] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:13:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 05:13:50.718714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4eeea-d231-405e-8dfb-c3ffe05d97e7 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:50.729575 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4eeea-d231-405e-8dfb-c3ffe05d97e7 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:50.734188 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb926a-a4f8-4ddb-bd9f-ef71c4803fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:50.734775 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb926a-a4f8-4ddb-bd9f-ef71c4803fc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:50.735122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb926a-a4f8-4ddb-bd9f-ef71c4803fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:50.735405 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb926a-a4f8-4ddb-bd9f-ef71c4803fc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:50.736063 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb926a-a4f8-4ddb-bd9f-ef71c4803fc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:50.738601 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb926a-a4f8-4ddb-bd9f-ef71c4803fc4 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:50.762268 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-9f06b79c-00da-486f-8c70-45207ed1678d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '293662ace54645c8bbdfcdf3a80aa8d8', 'request_id': 'req-9f06b79c-00da-486f-8c70-45207ed1678d'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:50.773680 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-9f06b79c-00da-486f-8c70-45207ed1678d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '293662ace54645c8bbdfcdf3a80aa8d8', 'request_id': 'req-9f06b79c-00da-486f-8c70-45207ed1678d'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:50.852926 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1000/2004] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:13:50 2022] POST /identity/v3/projects => generated 391 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:13:50.853885 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb926a-a4f8-4ddb-bd9f-ef71c4803fc4 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:50.864310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-9f06b79c-00da-486f-8c70-45207ed1678d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '293662ace54645c8bbdfcdf3a80aa8d8', 'request_id': 'req-9f06b79c-00da-486f-8c70-45207ed1678d'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:50.876061 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-9f06b79c-00da-486f-8c70-45207ed1678d admin admin] The token cache is being invalidate because project 293662ace54645c8bbdfcdf3a80aa8d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:13:50.876354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-9f06b79c-00da-486f-8c70-45207ed1678d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:50.877082 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-9f06b79c-00da-486f-8c70-45207ed1678d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:50.878652 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1005/2005] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:13:50 2022] DELETE /identity/v3/projects/293662ace54645c8bbdfcdf3a80aa8d8 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:13:50.883692 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a79210-b2d6-4a12-bcb6-1221f34f5b53 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:50.883981 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a79210-b2d6-4a12-bcb6-1221f34f5b53 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:50.884262 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a79210-b2d6-4a12-bcb6-1221f34f5b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:50.904180 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-61a79210-b2d6-4a12-bcb6-1221f34f5b53 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:50.905559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-61a79210-b2d6-4a12-bcb6-1221f34f5b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:50.916791 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb926a-a4f8-4ddb-bd9f-ef71c4803fc4 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:50.921913 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74de11b-7949-4cdd-ae36-06f30dd875e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:50.922544 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74de11b-7949-4cdd-ae36-06f30dd875e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:50.922829 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74de11b-7949-4cdd-ae36-06f30dd875e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:50.923138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74de11b-7949-4cdd-ae36-06f30dd875e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:50.924478 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74de11b-7949-4cdd-ae36-06f30dd875e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:50.926409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74de11b-7949-4cdd-ae36-06f30dd875e6 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:50.962443 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:50.967887 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1006/2006] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:13:50.976543 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef61eb8-2b7d-4c6b-be0b-5e76e40a717b None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:50.976827 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef61eb8-2b7d-4c6b-be0b-5e76e40a717b None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:50.977108 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef61eb8-2b7d-4c6b-be0b-5e76e40a717b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:50.999704 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-3ef61eb8-2b7d-4c6b-be0b-5e76e40a717b None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:51.001062 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-3ef61eb8-2b7d-4c6b-be0b-5e76e40a717b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:51.009000 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1007/2007] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:13:51.013032 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef61eb8-2b7d-4c6b-be0b-5e76e40a717b None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:51.022920 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef61eb8-2b7d-4c6b-be0b-5e76e40a717b None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:51.028269 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb129e5c-35db-49d9-a491-a292ff8afacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:51.029017 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb129e5c-35db-49d9-a491-a292ff8afacc admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:51.029305 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb129e5c-35db-49d9-a491-a292ff8afacc admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:51.029608 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb129e5c-35db-49d9-a491-a292ff8afacc admin admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:51.030316 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb129e5c-35db-49d9-a491-a292ff8afacc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:13:51.030719 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb129e5c-35db-49d9-a491-a292ff8afacc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:51.032375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb129e5c-35db-49d9-a491-a292ff8afacc admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:51.033131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1001/2008] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:13:50 2022] POST /identity/v3/users => generated 296 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:13:51.037634 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74de11b-7949-4cdd-ae36-06f30dd875e6 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:51.039722 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c74de11b-7949-4cdd-ae36-06f30dd875e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:51.049515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1008/2009] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 05:13:51.052336 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74de11b-7949-4cdd-ae36-06f30dd875e6 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:51.053877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb129e5c-35db-49d9-a491-a292ff8afacc admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:51.058080 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b04650-0f61-48db-b7da-8e29d8369d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:51.058717 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b04650-0f61-48db-b7da-8e29d8369d7a admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:51.059025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b04650-0f61-48db-b7da-8e29d8369d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:51.059311 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b04650-0f61-48db-b7da-8e29d8369d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:51.060045 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b04650-0f61-48db-b7da-8e29d8369d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:51.061672 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b04650-0f61-48db-b7da-8e29d8369d7a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:51.067158 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb129e5c-35db-49d9-a491-a292ff8afacc admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:51.073832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab87bbd-b610-47da-a38f-3cd57922c788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:51.074562 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab87bbd-b610-47da-a38f-3cd57922c788 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:51.074950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab87bbd-b610-47da-a38f-3cd57922c788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:51.075296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab87bbd-b610-47da-a38f-3cd57922c788 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:51.076101 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab87bbd-b610-47da-a38f-3cd57922c788 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:51.076529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1002/2010] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:13:51.078188 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab87bbd-b610-47da-a38f-3cd57922c788 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:51.081029 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b04650-0f61-48db-b7da-8e29d8369d7a admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:51.092498 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b04650-0f61-48db-b7da-8e29d8369d7a admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:51.098788 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8cda70-bc16-42e5-8cd6-2f9b077185a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:51.102292 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8cda70-bc16-42e5-8cd6-2f9b077185a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:51.102292 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8cda70-bc16-42e5-8cd6-2f9b077185a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:51.102292 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8cda70-bc16-42e5-8cd6-2f9b077185a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:51.102292 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8cda70-bc16-42e5-8cd6-2f9b077185a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:51.102292 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8cda70-bc16-42e5-8cd6-2f9b077185a7 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:51.113391 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1003/2011] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:51.117646 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8cda70-bc16-42e5-8cd6-2f9b077185a7 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:51.127958 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8cda70-bc16-42e5-8cd6-2f9b077185a7 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:51.132891 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ff676-6901-41bd-9045-aaa7cea2424c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:51.133540 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ff676-6901-41bd-9045-aaa7cea2424c admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:51.133849 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ff676-6901-41bd-9045-aaa7cea2424c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:51.134136 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ff676-6901-41bd-9045-aaa7cea2424c admin admin] PATH_INFO: `/v3/projects/f6f39d74ff2c4dde918584332b2aff3f/users/87674141f405461fb5671a60099760c4/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:51.150369 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ff676-6901-41bd-9045-aaa7cea2424c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6f39d74ff2c4dde918584332b2aff3f, user_id=87674141f405461fb5671a60099760c4, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:51.153074 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ff676-6901-41bd-9045-aaa7cea2424c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:51.190731 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1009/2012] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:13:51 2022] POST /identity/v3/projects => generated 375 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:13:51.192190 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab87bbd-b610-47da-a38f-3cd57922c788 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:51.258648 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1004/2013] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:51 2022] PUT /identity/v3/projects/f6f39d74ff2c4dde918584332b2aff3f/users/87674141f405461fb5671a60099760c4/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:13:51.284477 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15514c-48d3-456d-b6f2-cff7be708b3b None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:51.285834 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15514c-48d3-456d-b6f2-cff7be708b3b None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:51.285834 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15514c-48d3-456d-b6f2-cff7be708b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:51.289576 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab87bbd-b610-47da-a38f-3cd57922c788 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:51.320819 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-2a15514c-48d3-456d-b6f2-cff7be708b3b None None] MFA Rules not processed for user `87674141f405461fb5671a60099760c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:51.321994 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2a15514c-48d3-456d-b6f2-cff7be708b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:51.354712 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1bf1aa-35e2-4635-9706-221598f0e2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:51.355876 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bf1aa-35e2-4635-9706-221598f0e2a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:51.356233 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bf1aa-35e2-4635-9706-221598f0e2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:51.356528 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bf1aa-35e2-4635-9706-221598f0e2a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:51.359066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1bf1aa-35e2-4635-9706-221598f0e2a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:51.359731 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1bf1aa-35e2-4635-9706-221598f0e2a6 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:51.416745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1005/2014] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:51 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:51.421375 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15514c-48d3-456d-b6f2-cff7be708b3b None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:51.441831 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15514c-48d3-456d-b6f2-cff7be708b3b None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:51.446613 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce49197-5d0e-4baf-9fbd-c46afe029718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:51.447278 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce49197-5d0e-4baf-9fbd-c46afe029718 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:51.447588 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce49197-5d0e-4baf-9fbd-c46afe029718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:51.447847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce49197-5d0e-4baf-9fbd-c46afe029718 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:51.448527 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce49197-5d0e-4baf-9fbd-c46afe029718 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:51.450606 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce49197-5d0e-4baf-9fbd-c46afe029718 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:51.463230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1010/2015] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:13:51 2022] POST /identity/v3/users => generated 288 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:13:51.473242 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1bf1aa-35e2-4635-9706-221598f0e2a6 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:51.485196 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1bf1aa-35e2-4635-9706-221598f0e2a6 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:51.490180 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bcb997-2c2a-41be-b899-462bc0955b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:51.490965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bcb997-2c2a-41be-b899-462bc0955b5b admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:51.491252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bcb997-2c2a-41be-b899-462bc0955b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:51.491529 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bcb997-2c2a-41be-b899-462bc0955b5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:51.492252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bcb997-2c2a-41be-b899-462bc0955b5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:51.493893 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bcb997-2c2a-41be-b899-462bc0955b5b admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:51.504965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1011/2016] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:51.508829 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bcb997-2c2a-41be-b899-462bc0955b5b admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:51.519532 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bcb997-2c2a-41be-b899-462bc0955b5b admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:51.524868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c263384-e015-4272-9584-df50de826a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:51.525491 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c263384-e015-4272-9584-df50de826a12 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:51.525777 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c263384-e015-4272-9584-df50de826a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:51.526053 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c263384-e015-4272-9584-df50de826a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:51.526738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c263384-e015-4272-9584-df50de826a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:51.528605 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c263384-e015-4272-9584-df50de826a12 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:51.529510 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:51.539693 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1012/2017] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:13:51.543793 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c263384-e015-4272-9584-df50de826a12 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:51.554915 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c263384-e015-4272-9584-df50de826a12 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:51.559640 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab959c-7160-41d1-933c-4393059e481e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:51.560568 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab959c-7160-41d1-933c-4393059e481e admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:51.560568 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab959c-7160-41d1-933c-4393059e481e admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:51.560850 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab959c-7160-41d1-933c-4393059e481e admin admin] PATH_INFO: `/v3/projects/7b2aac05d35042f6b7e0603d59f7385f/users/27eaa7e3daee4b4587d8dad39bf4a42a/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:51.570921 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1006/2018] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:13:51 2022] POST /identity/v3/projects => generated 393 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:13:51.572419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce49197-5d0e-4baf-9fbd-c46afe029718 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:51.581892 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab959c-7160-41d1-933c-4393059e481e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b2aac05d35042f6b7e0603d59f7385f, user_id=27eaa7e3daee4b4587d8dad39bf4a42a, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:51.584835 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab959c-7160-41d1-933c-4393059e481e admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:51.639989 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce49197-5d0e-4baf-9fbd-c46afe029718 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:51.645322 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd98011-238f-4a0d-becc-122efb8d3a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:51.645975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd98011-238f-4a0d-becc-122efb8d3a05 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:51.646258 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd98011-238f-4a0d-becc-122efb8d3a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:51.646555 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd98011-238f-4a0d-becc-122efb8d3a05 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:51.647237 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd98011-238f-4a0d-becc-122efb8d3a05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:51.649168 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd98011-238f-4a0d-becc-122efb8d3a05 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:51.699817 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1013/2019] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:51 2022] PUT /identity/v3/projects/7b2aac05d35042f6b7e0603d59f7385f/users/27eaa7e3daee4b4587d8dad39bf4a42a/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:13:51.707877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec09475-1b02-45d4-b53d-8f77cd76fb87 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:51.708171 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec09475-1b02-45d4-b53d-8f77cd76fb87 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:51.708454 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec09475-1b02-45d4-b53d-8f77cd76fb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:51.745259 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-8ec09475-1b02-45d4-b53d-8f77cd76fb87 None None] MFA Rules not processed for user `27eaa7e3daee4b4587d8dad39bf4a42a`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:51.746637 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-8ec09475-1b02-45d4-b53d-8f77cd76fb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:51.787763 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1007/2020] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:13:51 2022] POST /identity/v3/users => generated 303 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:51.791844 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd98011-238f-4a0d-becc-122efb8d3a05 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:51.802368 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd98011-238f-4a0d-becc-122efb8d3a05 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:51.817001 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1014/2021] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:13:51.828965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec09475-1b02-45d4-b53d-8f77cd76fb87 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:51.830626 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-8ec09475-1b02-45d4-b53d-8f77cd76fb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:51.873945 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d46a15-4f7d-4f62-9bc1-831d97a39b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:51.874587 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d46a15-4f7d-4f62-9bc1-831d97a39b0c admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:51.874893 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d46a15-4f7d-4f62-9bc1-831d97a39b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:51.875893 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d46a15-4f7d-4f62-9bc1-831d97a39b0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:51.876615 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d46a15-4f7d-4f62-9bc1-831d97a39b0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:51.879769 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d46a15-4f7d-4f62-9bc1-831d97a39b0c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:51.894818 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1008/2022] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:51.898894 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d46a15-4f7d-4f62-9bc1-831d97a39b0c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:51.899975 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec09475-1b02-45d4-b53d-8f77cd76fb87 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:51.905032 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550dfae-974f-4dbe-89c8-56562a051d08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:51.905757 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550dfae-974f-4dbe-89c8-56562a051d08 service cinder] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:51.906051 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550dfae-974f-4dbe-89c8-56562a051d08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:51.906375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550dfae-974f-4dbe-89c8-56562a051d08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:51.914874 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d46a15-4f7d-4f62-9bc1-831d97a39b0c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:51.919727 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda88b4-67ef-49c0-9be2-a35eb7e9d299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:51.920408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda88b4-67ef-49c0-9be2-a35eb7e9d299 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:51.920692 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda88b4-67ef-49c0-9be2-a35eb7e9d299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:51.927866 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda88b4-67ef-49c0-9be2-a35eb7e9d299 admin admin] PATH_INFO: `/v3/projects/36e960bf23fc42708dcb8aeee17a3973/users/d881b1abff604a9183eb55485ce40e0d/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:51.929194 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8550dfae-974f-4dbe-89c8-56562a051d08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:51.930678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8550dfae-974f-4dbe-89c8-56562a051d08 service cinder] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:51.938881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdda88b4-67ef-49c0-9be2-a35eb7e9d299 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36e960bf23fc42708dcb8aeee17a3973, user_id=d881b1abff604a9183eb55485ce40e0d, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:51.942301 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1015/2023] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:51.942471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdda88b4-67ef-49c0-9be2-a35eb7e9d299 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:51.992672 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:52.062001 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1009/2024] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:51 2022] PUT /identity/v3/projects/36e960bf23fc42708dcb8aeee17a3973/users/d881b1abff604a9183eb55485ce40e0d/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:52.066100 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550dfae-974f-4dbe-89c8-56562a051d08 service cinder] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:52.078592 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550dfae-974f-4dbe-89c8-56562a051d08 service cinder] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:52.144563 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a54ffd-c1e3-43bb-829b-7580e5573e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:52.145804 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a54ffd-c1e3-43bb-829b-7580e5573e91 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:52.145804 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a54ffd-c1e3-43bb-829b-7580e5573e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:52.146005 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a54ffd-c1e3-43bb-829b-7580e5573e91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:52.147295 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a54ffd-c1e3-43bb-829b-7580e5573e91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:52.149213 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a54ffd-c1e3-43bb-829b-7580e5573e91 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:52.161496 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1016/2025] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:52.165772 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda88b4-67ef-49c0-9be2-a35eb7e9d299 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:52.177218 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda88b4-67ef-49c0-9be2-a35eb7e9d299 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:52.182169 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed6a686-02e4-466f-a59e-55afaef414f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:52.182868 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6a686-02e4-466f-a59e-55afaef414f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:52.183177 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6a686-02e4-466f-a59e-55afaef414f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:52.183458 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6a686-02e4-466f-a59e-55afaef414f5 admin admin] PATH_INFO: `/v3/projects/36e960bf23fc42708dcb8aeee17a3973/users/d881b1abff604a9183eb55485ce40e0d/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:52.185396 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed6a686-02e4-466f-a59e-55afaef414f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36e960bf23fc42708dcb8aeee17a3973, user_id=d881b1abff604a9183eb55485ce40e0d, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:52.188196 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed6a686-02e4-466f-a59e-55afaef414f5 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:52.199701 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1010/2026] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:52 2022] PUT /identity/v3/projects/36e960bf23fc42708dcb8aeee17a3973/users/d881b1abff604a9183eb55485ce40e0d/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:52.210613 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183eed3-0f54-4fd6-ae07-b4ff8ab1544d None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:52.211073 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183eed3-0f54-4fd6-ae07-b4ff8ab1544d None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:52.211469 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183eed3-0f54-4fd6-ae07-b4ff8ab1544d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:52.244517 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-4183eed3-0f54-4fd6-ae07-b4ff8ab1544d None None] MFA Rules not processed for user `d881b1abff604a9183eb55485ce40e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:52.245761 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-4183eed3-0f54-4fd6-ae07-b4ff8ab1544d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:52.326620 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1017/2027] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:52 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:52.340069 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed6a686-02e4-466f-a59e-55afaef414f5 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:52.352090 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9ed6a686-02e4-466f-a59e-55afaef414f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:52.423671 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed6a686-02e4-466f-a59e-55afaef414f5 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:52.427616 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ef015-e612-435f-b55f-fcb9b5b89334 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:52.428320 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ef015-e612-435f-b55f-fcb9b5b89334 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:52.428606 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ef015-e612-435f-b55f-fcb9b5b89334 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:52.428882 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ef015-e612-435f-b55f-fcb9b5b89334 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:52.438773 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ef015-e612-435f-b55f-fcb9b5b89334 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:52.440874 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ef015-e612-435f-b55f-fcb9b5b89334 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:52.451705 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1011/2028] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:52 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:52.502228 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4183eed3-0f54-4fd6-ae07-b4ff8ab1544d None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:52.514175 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4183eed3-0f54-4fd6-ae07-b4ff8ab1544d None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:52.520109 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea66fea-a352-4f82-a534-6a700ec0fa02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:52.520860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea66fea-a352-4f82-a534-6a700ec0fa02 service cinder] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:52.521249 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea66fea-a352-4f82-a534-6a700ec0fa02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:52.521647 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea66fea-a352-4f82-a534-6a700ec0fa02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:52.587891 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea66fea-a352-4f82-a534-6a700ec0fa02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:52.589346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea66fea-a352-4f82-a534-6a700ec0fa02 service cinder] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:52.599363 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1018/2029] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:52 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:52.738740 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:53.052444 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:53.727193 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ef015-e612-435f-b55f-fcb9b5b89334 service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:53.729903 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c63ef015-e612-435f-b55f-fcb9b5b89334 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:53.811589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ef015-e612-435f-b55f-fcb9b5b89334 service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:53.817000 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b0fc2c-6a51-448f-8a1a-ca3db3fc4401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:53.817683 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0fc2c-6a51-448f-8a1a-ca3db3fc4401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:53.817972 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0fc2c-6a51-448f-8a1a-ca3db3fc4401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:53.818265 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0fc2c-6a51-448f-8a1a-ca3db3fc4401 admin admin] PATH_INFO: `/v3/users/7b911e2bb4134ecd919be0287dd2db1f` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:53.819598 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b0fc2c-6a51-448f-8a1a-ca3db3fc4401 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b911e2bb4134ecd919be0287dd2db1f)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:53.821552 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b0fc2c-6a51-448f-8a1a-ca3db3fc4401 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:53.989137 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-09b0fc2c-6a51-448f-8a1a-ca3db3fc4401 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b911e2bb4134ecd919be0287dd2db1f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:53.996377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:54.060564 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-09b0fc2c-6a51-448f-8a1a-ca3db3fc4401 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b911e2bb4134ecd919be0287dd2db1f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:54.061219 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-09b0fc2c-6a51-448f-8a1a-ca3db3fc4401 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b911e2bb4134ecd919be0287dd2db1f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:54.074225 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-09b0fc2c-6a51-448f-8a1a-ca3db3fc4401 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b911e2bb4134ecd919be0287dd2db1f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:54.074904 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-09b0fc2c-6a51-448f-8a1a-ca3db3fc4401 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b911e2bb4134ecd919be0287dd2db1f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:54.087050 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1012/2030] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:13:53 2022] DELETE /identity/v3/users/7b911e2bb4134ecd919be0287dd2db1f => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:54.091735 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92c21a-7127-43f4-b6d2-b14c0e35559e None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:54.092189 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92c21a-7127-43f4-b6d2-b14c0e35559e None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:54.092304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92c21a-7127-43f4-b6d2-b14c0e35559e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:54.112223 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-0d92c21a-7127-43f4-b6d2-b14c0e35559e None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:54.113443 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-0d92c21a-7127-43f4-b6d2-b14c0e35559e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:54.173555 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1019/2031] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:54.183324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b0fc2c-6a51-448f-8a1a-ca3db3fc4401 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:54.184748 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-09b0fc2c-6a51-448f-8a1a-ca3db3fc4401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:54.241863 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b0fc2c-6a51-448f-8a1a-ca3db3fc4401 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:54.245524 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a736d-030e-46eb-8807-02e434f86c3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:54.246155 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a736d-030e-46eb-8807-02e434f86c3d service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:54.246442 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a736d-030e-46eb-8807-02e434f86c3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:54.246725 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a736d-030e-46eb-8807-02e434f86c3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:54.256071 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a736d-030e-46eb-8807-02e434f86c3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:54.257516 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a736d-030e-46eb-8807-02e434f86c3d service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:54.266687 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:54.269922 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1013/2032] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:54.312589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:54.671874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92c21a-7127-43f4-b6d2-b14c0e35559e None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:54.671874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-0d92c21a-7127-43f4-b6d2-b14c0e35559e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:54.682171 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92c21a-7127-43f4-b6d2-b14c0e35559e None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:54.687296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26797f85-d218-422a-86e1-5562b7d5f8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:54.687908 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26797f85-d218-422a-86e1-5562b7d5f8a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:54.688189 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26797f85-d218-422a-86e1-5562b7d5f8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:54.688464 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26797f85-d218-422a-86e1-5562b7d5f8a8 admin admin] PATH_INFO: `/v3/projects/95d4e8e5f33841b1b8fa3ec3c969d6d3` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:54.690193 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26797f85-d218-422a-86e1-5562b7d5f8a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95d4e8e5f33841b1b8fa3ec3c969d6d3)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:54.692162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26797f85-d218-422a-86e1-5562b7d5f8a8 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:54.761890 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:54.792548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:54.805147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-26797f85-d218-422a-86e1-5562b7d5f8a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95d4e8e5f33841b1b8fa3ec3c969d6d3', 'request_id': 'req-26797f85-d218-422a-86e1-5562b7d5f8a8'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:54.814973 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-26797f85-d218-422a-86e1-5562b7d5f8a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95d4e8e5f33841b1b8fa3ec3c969d6d3', 'request_id': 'req-26797f85-d218-422a-86e1-5562b7d5f8a8'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:54.893438 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-26797f85-d218-422a-86e1-5562b7d5f8a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95d4e8e5f33841b1b8fa3ec3c969d6d3', 'request_id': 'req-26797f85-d218-422a-86e1-5562b7d5f8a8'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:54.907084 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-26797f85-d218-422a-86e1-5562b7d5f8a8 admin admin] The token cache is being invalidate because project 95d4e8e5f33841b1b8fa3ec3c969d6d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:13:54.907084 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-26797f85-d218-422a-86e1-5562b7d5f8a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:54.907439 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-26797f85-d218-422a-86e1-5562b7d5f8a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:54.910197 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1020/2033] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:13:54 2022] DELETE /identity/v3/projects/95d4e8e5f33841b1b8fa3ec3c969d6d3 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:54.914296 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a736d-030e-46eb-8807-02e434f86c3d service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:54.926297 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a736d-030e-46eb-8807-02e434f86c3d service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:54.988121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1ce24-6054-4d2c-b847-914bfa915fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:54.988844 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1ce24-6054-4d2c-b847-914bfa915fd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:54.989258 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1ce24-6054-4d2c-b847-914bfa915fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:54.989628 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1ce24-6054-4d2c-b847-914bfa915fd6 admin admin] PATH_INFO: `/v3/users/dbe0ca1a1e544bc0a64d65b08dd96479` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:54.991056 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a1ce24-6054-4d2c-b847-914bfa915fd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbe0ca1a1e544bc0a64d65b08dd96479)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:54.992947 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a1ce24-6054-4d2c-b847-914bfa915fd6 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:55.193051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:55.201761 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e1a1ce24-6054-4d2c-b847-914bfa915fd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbe0ca1a1e544bc0a64d65b08dd96479'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:55.316337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e1a1ce24-6054-4d2c-b847-914bfa915fd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbe0ca1a1e544bc0a64d65b08dd96479'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:55.317554 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e1a1ce24-6054-4d2c-b847-914bfa915fd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbe0ca1a1e544bc0a64d65b08dd96479'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:55.333382 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e1a1ce24-6054-4d2c-b847-914bfa915fd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbe0ca1a1e544bc0a64d65b08dd96479'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:55.334457 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e1a1ce24-6054-4d2c-b847-914bfa915fd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbe0ca1a1e544bc0a64d65b08dd96479'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:55.348416 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1014/2034] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:13:54 2022] DELETE /identity/v3/users/dbe0ca1a1e544bc0a64d65b08dd96479 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:55.466313 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:55.838733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26797f85-d218-422a-86e1-5562b7d5f8a8 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:55.840296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-26797f85-d218-422a-86e1-5562b7d5f8a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:55.860218 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26797f85-d218-422a-86e1-5562b7d5f8a8 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:55.957414 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:55.964187 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1fa88f-5742-4f50-9a8b-f096452ad1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:55.965103 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fa88f-5742-4f50-9a8b-f096452ad1a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:55.965580 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fa88f-5742-4f50-9a8b-f096452ad1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:55.966318 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fa88f-5742-4f50-9a8b-f096452ad1a9 admin admin] PATH_INFO: `/v3/projects/e5a8e718f5dd40c293483a65c8f666d6` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:55.968149 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1fa88f-5742-4f50-9a8b-f096452ad1a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5a8e718f5dd40c293483a65c8f666d6)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:55.971195 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1fa88f-5742-4f50-9a8b-f096452ad1a9 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:55.987044 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:56.078777 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-cc1fa88f-5742-4f50-9a8b-f096452ad1a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5a8e718f5dd40c293483a65c8f666d6', 'request_id': 'req-cc1fa88f-5742-4f50-9a8b-f096452ad1a9'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:56.088333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-cc1fa88f-5742-4f50-9a8b-f096452ad1a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5a8e718f5dd40c293483a65c8f666d6', 'request_id': 'req-cc1fa88f-5742-4f50-9a8b-f096452ad1a9'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:56.158392 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-cc1fa88f-5742-4f50-9a8b-f096452ad1a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5a8e718f5dd40c293483a65c8f666d6', 'request_id': 'req-cc1fa88f-5742-4f50-9a8b-f096452ad1a9'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:56.174554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-cc1fa88f-5742-4f50-9a8b-f096452ad1a9 admin admin] The token cache is being invalidate because project e5a8e718f5dd40c293483a65c8f666d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:13:56.175117 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-cc1fa88f-5742-4f50-9a8b-f096452ad1a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:56.175871 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-cc1fa88f-5742-4f50-9a8b-f096452ad1a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:56.177741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1021/2035] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:13:55 2022] DELETE /identity/v3/projects/e5a8e718f5dd40c293483a65c8f666d6 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:56.182655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1baad9-b7d0-4506-b8cb-c9b66ad1e5d8 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:56.182981 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1baad9-b7d0-4506-b8cb-c9b66ad1e5d8 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:56.183265 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1baad9-b7d0-4506-b8cb-c9b66ad1e5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:56.204298 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-da1baad9-b7d0-4506-b8cb-c9b66ad1e5d8 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:56.205512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-da1baad9-b7d0-4506-b8cb-c9b66ad1e5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:56.267894 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1015/2036] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:56.275962 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbdd86-ceba-4346-93fd-140816c7b209 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:56.276263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbdd86-ceba-4346-93fd-140816c7b209 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:56.276544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbdd86-ceba-4346-93fd-140816c7b209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:56.296496 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-2ebbdd86-ceba-4346-93fd-140816c7b209 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:56.297726 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-2ebbdd86-ceba-4346-93fd-140816c7b209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:56.306069 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1022/2037] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:56.310244 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1baad9-b7d0-4506-b8cb-c9b66ad1e5d8 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:56.320448 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1baad9-b7d0-4506-b8cb-c9b66ad1e5d8 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:56.325202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07e60c-5364-44da-9a59-277a56f7ccfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:56.325823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07e60c-5364-44da-9a59-277a56f7ccfe admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:56.327853 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07e60c-5364-44da-9a59-277a56f7ccfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:56.327853 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07e60c-5364-44da-9a59-277a56f7ccfe admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:56.327853 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd07e60c-5364-44da-9a59-277a56f7ccfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:13:56.327853 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd07e60c-5364-44da-9a59-277a56f7ccfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:56.329095 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd07e60c-5364-44da-9a59-277a56f7ccfe admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:56.341870 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1016/2038] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:13:56.346911 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbdd86-ceba-4346-93fd-140816c7b209 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:56.356306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbdd86-ceba-4346-93fd-140816c7b209 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:56.361106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b068b6d-e4fd-4897-b099-d594e4eeb53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:56.361718 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b068b6d-e4fd-4897-b099-d594e4eeb53a admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:56.361998 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b068b6d-e4fd-4897-b099-d594e4eeb53a admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:56.362270 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b068b6d-e4fd-4897-b099-d594e4eeb53a admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:56.363037 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b068b6d-e4fd-4897-b099-d594e4eeb53a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:56.364922 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b068b6d-e4fd-4897-b099-d594e4eeb53a admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:56.495340 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1023/2039] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:13:56 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:13:56.499862 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07e60c-5364-44da-9a59-277a56f7ccfe admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:56.515637 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07e60c-5364-44da-9a59-277a56f7ccfe admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:56.574034 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515fe573-6785-490e-aba5-c1d713148f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:56.574657 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515fe573-6785-490e-aba5-c1d713148f1f admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:56.574985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515fe573-6785-490e-aba5-c1d713148f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:56.575264 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515fe573-6785-490e-aba5-c1d713148f1f admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:56.576024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515fe573-6785-490e-aba5-c1d713148f1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:56.578000 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515fe573-6785-490e-aba5-c1d713148f1f admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:56.607244 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:56.664730 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1017/2040] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:13:56 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:56.665978 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b068b6d-e4fd-4897-b099-d594e4eeb53a admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:56.676791 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b068b6d-e4fd-4897-b099-d594e4eeb53a admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:56.682025 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dacf98-12f0-4fac-8bd1-dd663eaf218f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:56.682650 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dacf98-12f0-4fac-8bd1-dd663eaf218f admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:56.682962 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dacf98-12f0-4fac-8bd1-dd663eaf218f admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:56.683241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dacf98-12f0-4fac-8bd1-dd663eaf218f admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:56.684049 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dacf98-12f0-4fac-8bd1-dd663eaf218f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:56.685693 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dacf98-12f0-4fac-8bd1-dd663eaf218f admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:56.696590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1024/2041] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:56.700817 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515fe573-6785-490e-aba5-c1d713148f1f admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:56.711735 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515fe573-6785-490e-aba5-c1d713148f1f admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:56.716586 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88c0ab-94c2-4310-a8b8-9c0aa9fd1500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:56.717201 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88c0ab-94c2-4310-a8b8-9c0aa9fd1500 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:56.717488 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88c0ab-94c2-4310-a8b8-9c0aa9fd1500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:56.717769 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88c0ab-94c2-4310-a8b8-9c0aa9fd1500 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:56.718468 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc88c0ab-94c2-4310-a8b8-9c0aa9fd1500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:56.720165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc88c0ab-94c2-4310-a8b8-9c0aa9fd1500 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:56.732094 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1018/2042] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:56.736822 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dacf98-12f0-4fac-8bd1-dd663eaf218f admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:56.747151 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dacf98-12f0-4fac-8bd1-dd663eaf218f admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:56.751897 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96356db-4933-423a-b833-a46feccbda54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:56.752641 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96356db-4933-423a-b833-a46feccbda54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:56.752923 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96356db-4933-423a-b833-a46feccbda54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:56.753198 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96356db-4933-423a-b833-a46feccbda54 admin admin] PATH_INFO: `/v3/projects/eb19dd0b1c704f4c9c38fa90677e5bb9/users/bec3e46ccb9e48cc9eb495b500250ce3/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:56.757749 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:56.769159 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96356db-4933-423a-b833-a46feccbda54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb19dd0b1c704f4c9c38fa90677e5bb9, user_id=bec3e46ccb9e48cc9eb495b500250ce3, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:56.772108 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96356db-4933-423a-b833-a46feccbda54 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:56.869420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1025/2043] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:56 2022] PUT /identity/v3/projects/eb19dd0b1c704f4c9c38fa90677e5bb9/users/bec3e46ccb9e48cc9eb495b500250ce3/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:56.879481 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91d3fe-933e-4734-9fef-b55de9ad2ec3 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:56.879775 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91d3fe-933e-4734-9fef-b55de9ad2ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:56.880053 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91d3fe-933e-4734-9fef-b55de9ad2ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:56.909558 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-ab91d3fe-933e-4734-9fef-b55de9ad2ec3 None None] MFA Rules not processed for user `bec3e46ccb9e48cc9eb495b500250ce3`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:56.910784 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ab91d3fe-933e-4734-9fef-b55de9ad2ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:56.976651 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1019/2044] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:56.984371 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:56.988992 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96356db-4933-423a-b833-a46feccbda54 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:56.991050 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-a96356db-4933-423a-b833-a46feccbda54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:57.007541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:57.032396 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:57.055418 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96356db-4933-423a-b833-a46feccbda54 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:57.059120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22afc613-20a8-4cec-aa9a-0d4c89460722 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:57.059828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22afc613-20a8-4cec-aa9a-0d4c89460722 service cinder] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:57.060112 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22afc613-20a8-4cec-aa9a-0d4c89460722 service cinder] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:57.060381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22afc613-20a8-4cec-aa9a-0d4c89460722 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:57.069868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22afc613-20a8-4cec-aa9a-0d4c89460722 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:57.071366 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22afc613-20a8-4cec-aa9a-0d4c89460722 service cinder] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:57.080659 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1026/2045] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:57.100134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91d3fe-933e-4734-9fef-b55de9ad2ec3 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:57.164719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91d3fe-933e-4734-9fef-b55de9ad2ec3 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:57.169380 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aff502-d529-488c-88b3-58d408578f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:57.170026 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aff502-d529-488c-88b3-58d408578f7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:57.170316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aff502-d529-488c-88b3-58d408578f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:57.170605 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aff502-d529-488c-88b3-58d408578f7b admin admin] PATH_INFO: `/v3/users/bec3e46ccb9e48cc9eb495b500250ce3` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:57.171892 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aff502-d529-488c-88b3-58d408578f7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bec3e46ccb9e48cc9eb495b500250ce3)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:57.174415 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aff502-d529-488c-88b3-58d408578f7b admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:57.224925 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:57.432020 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:57.434978 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-55aff502-d529-488c-88b3-58d408578f7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bec3e46ccb9e48cc9eb495b500250ce3'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:57.500346 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-55aff502-d529-488c-88b3-58d408578f7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bec3e46ccb9e48cc9eb495b500250ce3'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:57.500989 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-55aff502-d529-488c-88b3-58d408578f7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bec3e46ccb9e48cc9eb495b500250ce3'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:57.513368 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-55aff502-d529-488c-88b3-58d408578f7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bec3e46ccb9e48cc9eb495b500250ce3'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:57.514000 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-55aff502-d529-488c-88b3-58d408578f7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bec3e46ccb9e48cc9eb495b500250ce3'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:57.526995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1020/2046] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:13:57 2022] DELETE /identity/v3/users/bec3e46ccb9e48cc9eb495b500250ce3 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:57.529216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:57.531670 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32843ee-67f2-4066-b984-187cff149469 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:57.532272 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32843ee-67f2-4066-b984-187cff149469 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:57.532272 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32843ee-67f2-4066-b984-187cff149469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:57.553128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-f32843ee-67f2-4066-b984-187cff149469 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:57.554385 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-f32843ee-67f2-4066-b984-187cff149469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:57.617623 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1027/2047] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:57.627875 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aff502-d529-488c-88b3-58d408578f7b admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:57.629372 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-55aff502-d529-488c-88b3-58d408578f7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:57.691458 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aff502-d529-488c-88b3-58d408578f7b admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:57.695225 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638bb879-1026-4e40-b08b-4708922dcd54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:57.695875 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bb879-1026-4e40-b08b-4708922dcd54 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:57.696180 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bb879-1026-4e40-b08b-4708922dcd54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:57.696450 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bb879-1026-4e40-b08b-4708922dcd54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:57.705714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638bb879-1026-4e40-b08b-4708922dcd54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:57.707177 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638bb879-1026-4e40-b08b-4708922dcd54 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:57.717229 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1021/2048] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:57.775229 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:57.849601 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:58.194149 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32843ee-67f2-4066-b984-187cff149469 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:58.195825 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-f32843ee-67f2-4066-b984-187cff149469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:58.216284 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:58.229790 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32843ee-67f2-4066-b984-187cff149469 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:58.230060 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9724f7a4-190d-41b7-bceb-bb164cbe55a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:58.231103 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724f7a4-190d-41b7-bceb-bb164cbe55a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:58.231103 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724f7a4-190d-41b7-bceb-bb164cbe55a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:58.231316 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724f7a4-190d-41b7-bceb-bb164cbe55a0 admin admin] PATH_INFO: `/v3/projects/eb19dd0b1c704f4c9c38fa90677e5bb9` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:58.232524 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9724f7a4-190d-41b7-bceb-bb164cbe55a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb19dd0b1c704f4c9c38fa90677e5bb9)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:58.234530 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9724f7a4-190d-41b7-bceb-bb164cbe55a0 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:58.358250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-9724f7a4-190d-41b7-bceb-bb164cbe55a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb19dd0b1c704f4c9c38fa90677e5bb9', 'request_id': 'req-9724f7a4-190d-41b7-bceb-bb164cbe55a0'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:58.365427 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-9724f7a4-190d-41b7-bceb-bb164cbe55a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb19dd0b1c704f4c9c38fa90677e5bb9', 'request_id': 'req-9724f7a4-190d-41b7-bceb-bb164cbe55a0'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:58.464214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-9724f7a4-190d-41b7-bceb-bb164cbe55a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb19dd0b1c704f4c9c38fa90677e5bb9', 'request_id': 'req-9724f7a4-190d-41b7-bceb-bb164cbe55a0'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:58.478306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-9724f7a4-190d-41b7-bceb-bb164cbe55a0 admin admin] The token cache is being invalidate because project eb19dd0b1c704f4c9c38fa90677e5bb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:13:58.479056 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-9724f7a4-190d-41b7-bceb-bb164cbe55a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:58.480426 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-9724f7a4-190d-41b7-bceb-bb164cbe55a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:58.487462 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b4532-4ce4-44de-b940-893add2279ea None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:58.487996 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b4532-4ce4-44de-b940-893add2279ea None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:58.488247 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b4532-4ce4-44de-b940-893add2279ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:58.491587 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1028/2049] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:13:58 2022] DELETE /identity/v3/projects/eb19dd0b1c704f4c9c38fa90677e5bb9 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:58.510242 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-d11b4532-4ce4-44de-b940-893add2279ea None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:58.512128 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d11b4532-4ce4-44de-b940-893add2279ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:58.621671 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1022/2050] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:58.631414 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb51d30-9ad1-4f9b-8d23-3f11f63f4159 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:58.632023 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb51d30-9ad1-4f9b-8d23-3f11f63f4159 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:58.632023 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb51d30-9ad1-4f9b-8d23-3f11f63f4159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:58.655761 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:58.672732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-adb51d30-9ad1-4f9b-8d23-3f11f63f4159 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:58.674825 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-adb51d30-9ad1-4f9b-8d23-3f11f63f4159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:58.687782 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1029/2051] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:58.689559 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b4532-4ce4-44de-b940-893add2279ea None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:58.701801 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b4532-4ce4-44de-b940-893add2279ea None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:58.707142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf10e56-5d74-460e-b6f3-cfb0cf883782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:58.707770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf10e56-5d74-460e-b6f3-cfb0cf883782 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:58.708887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf10e56-5d74-460e-b6f3-cfb0cf883782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:58.708887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf10e56-5d74-460e-b6f3-cfb0cf883782 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:58.709279 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf10e56-5d74-460e-b6f3-cfb0cf883782 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:13:58.709589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf10e56-5d74-460e-b6f3-cfb0cf883782 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:58.713355 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf10e56-5d74-460e-b6f3-cfb0cf883782 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:58.728168 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1023/2052] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:13:58.733055 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb51d30-9ad1-4f9b-8d23-3f11f63f4159 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:58.744809 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb51d30-9ad1-4f9b-8d23-3f11f63f4159 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:58.753514 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5487b9-0595-4960-886b-2e9c3fe285ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:58.758507 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5487b9-0595-4960-886b-2e9c3fe285ed admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:58.758507 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5487b9-0595-4960-886b-2e9c3fe285ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:58.758507 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5487b9-0595-4960-886b-2e9c3fe285ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:58.758507 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5487b9-0595-4960-886b-2e9c3fe285ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:58.758507 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5487b9-0595-4960-886b-2e9c3fe285ed admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:58.899680 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1030/2053] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:13:58 2022] POST /identity/v3/projects => generated 369 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:13:58.900797 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf10e56-5d74-460e-b6f3-cfb0cf883782 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:58.912396 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf10e56-5d74-460e-b6f3-cfb0cf883782 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:58.993227 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cfb7d2-9d0a-4cb1-b2cf-2831650021a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:58.993972 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cfb7d2-9d0a-4cb1-b2cf-2831650021a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:58.994338 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cfb7d2-9d0a-4cb1-b2cf-2831650021a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:58.994693 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cfb7d2-9d0a-4cb1-b2cf-2831650021a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:58.995491 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cfb7d2-9d0a-4cb1-b2cf-2831650021a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:58.997520 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cfb7d2-9d0a-4cb1-b2cf-2831650021a4 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:59.107996 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1024/2054] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:13:58 2022] POST /identity/v3/users => generated 285 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:13:59.112669 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5487b9-0595-4960-886b-2e9c3fe285ed admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:59.125217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5487b9-0595-4960-886b-2e9c3fe285ed admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:59.133321 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60034a3-bcb7-4e08-8c0d-2d5f45d343ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:59.134134 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60034a3-bcb7-4e08-8c0d-2d5f45d343ec admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:59.134535 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60034a3-bcb7-4e08-8c0d-2d5f45d343ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:59.135001 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60034a3-bcb7-4e08-8c0d-2d5f45d343ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:59.136051 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60034a3-bcb7-4e08-8c0d-2d5f45d343ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:59.160270 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60034a3-bcb7-4e08-8c0d-2d5f45d343ec admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:59.183354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1031/2055] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:59.187804 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cfb7d2-9d0a-4cb1-b2cf-2831650021a4 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:59.202011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cfb7d2-9d0a-4cb1-b2cf-2831650021a4 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:59.210377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a394e-9dfa-4aa8-90c2-1bab4585b29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:59.211206 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a394e-9dfa-4aa8-90c2-1bab4585b29f admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:59.211561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a394e-9dfa-4aa8-90c2-1bab4585b29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:59.211909 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a394e-9dfa-4aa8-90c2-1bab4585b29f admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:59.212763 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a394e-9dfa-4aa8-90c2-1bab4585b29f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:59.214782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a394e-9dfa-4aa8-90c2-1bab4585b29f admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:59.228955 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1025/2056] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:13:59.232957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60034a3-bcb7-4e08-8c0d-2d5f45d343ec admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:59.245957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60034a3-bcb7-4e08-8c0d-2d5f45d343ec admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:59.248514 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:59.253869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac314570-6e37-40b9-af4b-cc55133562f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:59.261146 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac314570-6e37-40b9-af4b-cc55133562f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:59.261146 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac314570-6e37-40b9-af4b-cc55133562f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:59.261146 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac314570-6e37-40b9-af4b-cc55133562f4 admin admin] PATH_INFO: `/v3/projects/d70415ca011a4138862bdcf9ba6e5b84/users/55dfb2ea79e44c4493dea3993ecd3018/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:59.272564 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac314570-6e37-40b9-af4b-cc55133562f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d70415ca011a4138862bdcf9ba6e5b84, user_id=55dfb2ea79e44c4493dea3993ecd3018, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:59.278079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac314570-6e37-40b9-af4b-cc55133562f4 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:59.337228 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a394e-9dfa-4aa8-90c2-1bab4585b29f admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:59.338881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a12a394e-9dfa-4aa8-90c2-1bab4585b29f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:59.353151 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a394e-9dfa-4aa8-90c2-1bab4585b29f admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:59.358822 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33975e33-b463-4244-83b7-f7e653b32d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:59.358822 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33975e33-b463-4244-83b7-f7e653b32d6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:59.358822 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33975e33-b463-4244-83b7-f7e653b32d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:59.359129 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33975e33-b463-4244-83b7-f7e653b32d6a admin admin] PATH_INFO: `/v3/users/87674141f405461fb5671a60099760c4` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:59.360264 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33975e33-b463-4244-83b7-f7e653b32d6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=87674141f405461fb5671a60099760c4)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:59.362743 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33975e33-b463-4244-83b7-f7e653b32d6a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:59.395095 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1032/2057] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:13:59 2022] PUT /identity/v3/projects/d70415ca011a4138862bdcf9ba6e5b84/users/55dfb2ea79e44c4493dea3993ecd3018/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:13:59.405315 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c68d6-d93e-48d3-b1eb-4bc9473dc662 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:59.405641 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c68d6-d93e-48d3-b1eb-4bc9473dc662 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:59.405924 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c68d6-d93e-48d3-b1eb-4bc9473dc662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:59.442149 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-4e3c68d6-d93e-48d3-b1eb-4bc9473dc662 None None] MFA Rules not processed for user `55dfb2ea79e44c4493dea3993ecd3018`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:59.443571 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-4e3c68d6-d93e-48d3-b1eb-4bc9473dc662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:59.519357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1033/2058] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:59.520680 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14005ac4-fcee-450d-b9eb-ece960a07da7 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:59.520967 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14005ac4-fcee-450d-b9eb-ece960a07da7 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:59.521252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14005ac4-fcee-450d-b9eb-ece960a07da7 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:59.522727 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1034/2059] 10.222.0.40 () {54 vars in 921 bytes} [Wed Aug 3 05:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:13:59.527896 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63f9d4-5d32-4125-9325-a13c8d3d4641 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:59.528933 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63f9d4-5d32-4125-9325-a13c8d3d4641 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:59.529354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63f9d4-5d32-4125-9325-a13c8d3d4641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:59.547707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-33975e33-b463-4244-83b7-f7e653b32d6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87674141f405461fb5671a60099760c4'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:59.552839 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-db63f9d4-5d32-4125-9325-a13c8d3d4641 None None] MFA Rules not processed for user `59846d88f4314c2382aa5cb8bae5053b`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:59.554086 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-db63f9d4-5d32-4125-9325-a13c8d3d4641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:59.618833 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-33975e33-b463-4244-83b7-f7e653b32d6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87674141f405461fb5671a60099760c4'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:59.619659 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-33975e33-b463-4244-83b7-f7e653b32d6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87674141f405461fb5671a60099760c4'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:59.622341 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1035/2060] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Aug 3 05:13:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:13:59.623307 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db63f9d4-5d32-4125-9325-a13c8d3d4641 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:59.627139 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-db63f9d4-5d32-4125-9325-a13c8d3d4641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:59.636609 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-33975e33-b463-4244-83b7-f7e653b32d6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87674141f405461fb5671a60099760c4'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:59.637477 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-33975e33-b463-4244-83b7-f7e653b32d6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87674141f405461fb5671a60099760c4'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:13:59.652302 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1026/2061] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:13:59 2022] DELETE /identity/v3/users/87674141f405461fb5671a60099760c4 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:13:59.653247 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33975e33-b463-4244-83b7-f7e653b32d6a admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:59.655577 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-33975e33-b463-4244-83b7-f7e653b32d6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:59.709469 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db63f9d4-5d32-4125-9325-a13c8d3d4641 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:59.771371 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d28fb-7187-489b-a418-a99f8dc2833a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cc14b33895443b893fab487bfe1858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:59.772016 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d28fb-7187-489b-a418-a99f8dc2833a service glance] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:59.773093 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d28fb-7187-489b-a418-a99f8dc2833a service glance] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:59.773268 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33975e33-b463-4244-83b7-f7e653b32d6a admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:59.773524 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d28fb-7187-489b-a418-a99f8dc2833a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:59.775706 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-d00d28fb-7187-489b-a418-a99f8dc2833a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:59.789246 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11c91d2-6c77-4bbe-8ce6-edce9965d771 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59846d88f4314c2382aa5cb8bae5053b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:59.789246 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c91d2-6c77-4bbe-8ce6-edce9965d771 service placement] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:59.789246 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c91d2-6c77-4bbe-8ce6-edce9965d771 service placement] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:59.789246 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c91d2-6c77-4bbe-8ce6-edce9965d771 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:59.789698 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d11c91d2-6c77-4bbe-8ce6-edce9965d771 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:59.796219 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11c91d2-6c77-4bbe-8ce6-edce9965d771 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:59.801178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11c91d2-6c77-4bbe-8ce6-edce9965d771 service placement] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:59.810147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1027/2062] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:59.811580 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b41b1-a291-40b8-9c09-5ec580928c3d None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:59.811940 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b41b1-a291-40b8-9c09-5ec580928c3d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:59.812363 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b41b1-a291-40b8-9c09-5ec580928c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:13:59.837001 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-633b41b1-a291-40b8-9c09-5ec580928c3d None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:13:59.838422 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-633b41b1-a291-40b8-9c09-5ec580928c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:59.862439 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d28fb-7187-489b-a418-a99f8dc2833a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:13:59.863963 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d28fb-7187-489b-a418-a99f8dc2833a service glance] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:13:59.880353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1036/2063] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:13:59.915314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1028/2064] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:13:59.925744 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d28fb-7187-489b-a418-a99f8dc2833a service glance] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:13:59.927875 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-d00d28fb-7187-489b-a418-a99f8dc2833a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:13:59.939275 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:13:59.993022 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d28fb-7187-489b-a418-a99f8dc2833a service glance] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:13:59.997666 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9563e0af-910c-4edc-a683-50677aac472c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:13:59.998343 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563e0af-910c-4edc-a683-50677aac472c service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:13:59.999248 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563e0af-910c-4edc-a683-50677aac472c service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:13:59.999538 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563e0af-910c-4edc-a683-50677aac472c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:00.016110 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9563e0af-910c-4edc-a683-50677aac472c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:00.017707 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9563e0af-910c-4edc-a683-50677aac472c service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:00.028983 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1037/2065] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:13:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:14:00.404771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b41b1-a291-40b8-9c09-5ec580928c3d None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:00.406203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-633b41b1-a291-40b8-9c09-5ec580928c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:00.423761 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b41b1-a291-40b8-9c09-5ec580928c3d None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:00.428607 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9bd67-b05b-4830-8c5c-f43c50344437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:00.429325 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9bd67-b05b-4830-8c5c-f43c50344437 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:00.429616 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9bd67-b05b-4830-8c5c-f43c50344437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:00.429898 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9bd67-b05b-4830-8c5c-f43c50344437 admin admin] PATH_INFO: `/v3/users/72fea87f31584c6cba9523d52ed7bbd9` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:00.431114 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d9bd67-b05b-4830-8c5c-f43c50344437 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72fea87f31584c6cba9523d52ed7bbd9)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:00.433053 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d9bd67-b05b-4830-8c5c-f43c50344437 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:00.471785 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9563e0af-910c-4edc-a683-50677aac472c service neutron] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:00.473486 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-9563e0af-910c-4edc-a683-50677aac472c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:00.487211 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9563e0af-910c-4edc-a683-50677aac472c service neutron] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:00.490197 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:00.494067 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff45d6d-509a-4e72-886b-4a12fa612f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:00.494720 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff45d6d-509a-4e72-886b-4a12fa612f00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:00.495053 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff45d6d-509a-4e72-886b-4a12fa612f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:00.500894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff45d6d-509a-4e72-886b-4a12fa612f00 admin admin] PATH_INFO: `/v3/projects/f6f39d74ff2c4dde918584332b2aff3f` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:00.500894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff45d6d-509a-4e72-886b-4a12fa612f00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6f39d74ff2c4dde918584332b2aff3f)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:00.500894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff45d6d-509a-4e72-886b-4a12fa612f00 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:00.675216 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-4ff45d6d-509a-4e72-886b-4a12fa612f00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6f39d74ff2c4dde918584332b2aff3f', 'request_id': 'req-4ff45d6d-509a-4e72-886b-4a12fa612f00'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:00.680602 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-53d9bd67-b05b-4830-8c5c-f43c50344437 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72fea87f31584c6cba9523d52ed7bbd9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:00.681911 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-4ff45d6d-509a-4e72-886b-4a12fa612f00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6f39d74ff2c4dde918584332b2aff3f', 'request_id': 'req-4ff45d6d-509a-4e72-886b-4a12fa612f00'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:00.741303 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-53d9bd67-b05b-4830-8c5c-f43c50344437 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72fea87f31584c6cba9523d52ed7bbd9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:00.742255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-53d9bd67-b05b-4830-8c5c-f43c50344437 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72fea87f31584c6cba9523d52ed7bbd9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:00.756012 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-53d9bd67-b05b-4830-8c5c-f43c50344437 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72fea87f31584c6cba9523d52ed7bbd9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:00.756714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-53d9bd67-b05b-4830-8c5c-f43c50344437 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72fea87f31584c6cba9523d52ed7bbd9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:00.770309 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1029/2066] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:14:00 2022] DELETE /identity/v3/users/72fea87f31584c6cba9523d52ed7bbd9 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:14:00.826225 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-4ff45d6d-509a-4e72-886b-4a12fa612f00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6f39d74ff2c4dde918584332b2aff3f', 'request_id': 'req-4ff45d6d-509a-4e72-886b-4a12fa612f00'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:00.840726 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-4ff45d6d-509a-4e72-886b-4a12fa612f00 admin admin] The token cache is being invalidate because project f6f39d74ff2c4dde918584332b2aff3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:14:00.841034 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-4ff45d6d-509a-4e72-886b-4a12fa612f00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:00.841724 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-4ff45d6d-509a-4e72-886b-4a12fa612f00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:00.843857 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1038/2067] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:14:00 2022] DELETE /identity/v3/projects/f6f39d74ff2c4dde918584332b2aff3f => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:14:00.848554 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9bd67-b05b-4830-8c5c-f43c50344437 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:00.850469 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-53d9bd67-b05b-4830-8c5c-f43c50344437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:00.930373 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9bd67-b05b-4830-8c5c-f43c50344437 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:00.936174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e9f98-1592-4f5a-a35a-956ce0342344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:00.937263 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e9f98-1592-4f5a-a35a-956ce0342344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:00.937427 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e9f98-1592-4f5a-a35a-956ce0342344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:00.938486 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e9f98-1592-4f5a-a35a-956ce0342344 admin admin] PATH_INFO: `/v3/users/d881b1abff604a9183eb55485ce40e0d` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:00.939781 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e9f98-1592-4f5a-a35a-956ce0342344 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d881b1abff604a9183eb55485ce40e0d)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:00.941742 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e9f98-1592-4f5a-a35a-956ce0342344 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:01.056700 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff45d6d-509a-4e72-886b-4a12fa612f00 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:01.063632 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-4ff45d6d-509a-4e72-886b-4a12fa612f00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:01.132770 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff45d6d-509a-4e72-886b-4a12fa612f00 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:01.137457 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa8823-76eb-41a2-9956-dd1f31f9f309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:01.138125 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa8823-76eb-41a2-9956-dd1f31f9f309 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:01.138405 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa8823-76eb-41a2-9956-dd1f31f9f309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:01.138684 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa8823-76eb-41a2-9956-dd1f31f9f309 admin admin] PATH_INFO: `/v3/projects/4c56eb76c3814e488d8447fc18dae874` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:01.139890 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa8823-76eb-41a2-9956-dd1f31f9f309 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c56eb76c3814e488d8447fc18dae874)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:01.142343 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa8823-76eb-41a2-9956-dd1f31f9f309 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:01.171702 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-900e9f98-1592-4f5a-a35a-956ce0342344 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd881b1abff604a9183eb55485ce40e0d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:01.191757 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:DELETE=5,keystone:SELECT=89,keystone:INSERT=6 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:01.288975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-900e9f98-1592-4f5a-a35a-956ce0342344 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd881b1abff604a9183eb55485ce40e0d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:01.289523 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-cafa8823-76eb-41a2-9956-dd1f31f9f309 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c56eb76c3814e488d8447fc18dae874', 'request_id': 'req-cafa8823-76eb-41a2-9956-dd1f31f9f309'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:01.289659 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-900e9f98-1592-4f5a-a35a-956ce0342344 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd881b1abff604a9183eb55485ce40e0d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:01.295927 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-cafa8823-76eb-41a2-9956-dd1f31f9f309 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c56eb76c3814e488d8447fc18dae874', 'request_id': 'req-cafa8823-76eb-41a2-9956-dd1f31f9f309'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:01.302522 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-900e9f98-1592-4f5a-a35a-956ce0342344 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd881b1abff604a9183eb55485ce40e0d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:01.303223 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-900e9f98-1592-4f5a-a35a-956ce0342344 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd881b1abff604a9183eb55485ce40e0d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:01.316087 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1030/2068] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:14:00 2022] DELETE /identity/v3/users/d881b1abff604a9183eb55485ce40e0d => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:14:01.391578 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-cafa8823-76eb-41a2-9956-dd1f31f9f309 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c56eb76c3814e488d8447fc18dae874', 'request_id': 'req-cafa8823-76eb-41a2-9956-dd1f31f9f309'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:01.403774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-cafa8823-76eb-41a2-9956-dd1f31f9f309 admin admin] The token cache is being invalidate because project 4c56eb76c3814e488d8447fc18dae874 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:14:01.404097 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-cafa8823-76eb-41a2-9956-dd1f31f9f309 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:01.404745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-cafa8823-76eb-41a2-9956-dd1f31f9f309 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:01.406280 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1039/2069] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:14:01 2022] DELETE /identity/v3/projects/4c56eb76c3814e488d8447fc18dae874 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:01.410406 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e9f98-1592-4f5a-a35a-956ce0342344 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:01.412267 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-900e9f98-1592-4f5a-a35a-956ce0342344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:01.477919 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e9f98-1592-4f5a-a35a-956ce0342344 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:01.482631 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e0d1f-9812-4a99-9c97-fa656ca71e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:01.483315 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e0d1f-9812-4a99-9c97-fa656ca71e1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:01.483603 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e0d1f-9812-4a99-9c97-fa656ca71e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:01.483884 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e0d1f-9812-4a99-9c97-fa656ca71e1c admin admin] PATH_INFO: `/v3/users/234007a4cd97466c894a6cf821856bd9` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:01.485063 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e0d1f-9812-4a99-9c97-fa656ca71e1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=234007a4cd97466c894a6cf821856bd9)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:01.487225 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e0d1f-9812-4a99-9c97-fa656ca71e1c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:01.659042 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-dc8e0d1f-9812-4a99-9c97-fa656ca71e1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '234007a4cd97466c894a6cf821856bd9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:01.707275 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa8823-76eb-41a2-9956-dd1f31f9f309 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:01.707275 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-cafa8823-76eb-41a2-9956-dd1f31f9f309 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:01.728976 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-dc8e0d1f-9812-4a99-9c97-fa656ca71e1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '234007a4cd97466c894a6cf821856bd9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:01.729569 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-dc8e0d1f-9812-4a99-9c97-fa656ca71e1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '234007a4cd97466c894a6cf821856bd9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:01.743251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-dc8e0d1f-9812-4a99-9c97-fa656ca71e1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '234007a4cd97466c894a6cf821856bd9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:01.743986 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-dc8e0d1f-9812-4a99-9c97-fa656ca71e1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '234007a4cd97466c894a6cf821856bd9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:01.759343 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1031/2070] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:14:01 2022] DELETE /identity/v3/users/234007a4cd97466c894a6cf821856bd9 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:01.772646 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa8823-76eb-41a2-9956-dd1f31f9f309 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:01.804971 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:01.843338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ac3ac-302a-498a-8923-10591ef16670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:01.844008 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ac3ac-302a-498a-8923-10591ef16670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:01.844283 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ac3ac-302a-498a-8923-10591ef16670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:01.844561 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ac3ac-302a-498a-8923-10591ef16670 admin admin] PATH_INFO: `/v3/projects/36e960bf23fc42708dcb8aeee17a3973` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:01.846577 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8ac3ac-302a-498a-8923-10591ef16670 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36e960bf23fc42708dcb8aeee17a3973)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:01.848905 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8ac3ac-302a-498a-8923-10591ef16670 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:01.959371 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-9d8ac3ac-302a-498a-8923-10591ef16670 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36e960bf23fc42708dcb8aeee17a3973', 'request_id': 'req-9d8ac3ac-302a-498a-8923-10591ef16670'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:01.965511 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-9d8ac3ac-302a-498a-8923-10591ef16670 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36e960bf23fc42708dcb8aeee17a3973', 'request_id': 'req-9d8ac3ac-302a-498a-8923-10591ef16670'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:02.037792 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e0d1f-9812-4a99-9c97-fa656ca71e1c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:02.039333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-dc8e0d1f-9812-4a99-9c97-fa656ca71e1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:02.052517 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-9d8ac3ac-302a-498a-8923-10591ef16670 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36e960bf23fc42708dcb8aeee17a3973', 'request_id': 'req-9d8ac3ac-302a-498a-8923-10591ef16670'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:02.054874 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e0d1f-9812-4a99-9c97-fa656ca71e1c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:02.067353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-9d8ac3ac-302a-498a-8923-10591ef16670 admin admin] The token cache is being invalidate because project 36e960bf23fc42708dcb8aeee17a3973 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:14:02.067353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-9d8ac3ac-302a-498a-8923-10591ef16670 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:02.067353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-9d8ac3ac-302a-498a-8923-10591ef16670 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:02.067353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1040/2071] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:14:01 2022] DELETE /identity/v3/projects/36e960bf23fc42708dcb8aeee17a3973 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:14:02.072080 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d3cef-c75d-40f8-a323-344be60a5bc8 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:02.072374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d3cef-c75d-40f8-a323-344be60a5bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:02.073286 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d3cef-c75d-40f8-a323-344be60a5bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:02.093147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-540d3cef-c75d-40f8-a323-344be60a5bc8 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:02.094799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-540d3cef-c75d-40f8-a323-344be60a5bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:02.135327 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a3b1e-f21e-4cc2-9b4f-8b50959871ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:02.135977 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a3b1e-f21e-4cc2-9b4f-8b50959871ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:02.136264 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a3b1e-f21e-4cc2-9b4f-8b50959871ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:02.136563 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a3b1e-f21e-4cc2-9b4f-8b50959871ba admin admin] PATH_INFO: `/v3/projects/882275428a70495caeab73663243f476` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:02.138017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a3b1e-f21e-4cc2-9b4f-8b50959871ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=882275428a70495caeab73663243f476)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:02.139716 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a3b1e-f21e-4cc2-9b4f-8b50959871ba admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:02.167026 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1041/2072] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:14:02.175496 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f479215-0c9f-4b7e-ba38-5c3722fe8f51 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:02.175784 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f479215-0c9f-4b7e-ba38-5c3722fe8f51 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:02.176062 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f479215-0c9f-4b7e-ba38-5c3722fe8f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:02.196602 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-4f479215-0c9f-4b7e-ba38-5c3722fe8f51 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:02.197814 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-4f479215-0c9f-4b7e-ba38-5c3722fe8f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:02.206664 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1042/2073] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:14:02.210859 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f479215-0c9f-4b7e-ba38-5c3722fe8f51 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:02.220180 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f479215-0c9f-4b7e-ba38-5c3722fe8f51 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:02.225198 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c68e3d-e552-48e4-a60c-d1a5052ce510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:02.225820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c68e3d-e552-48e4-a60c-d1a5052ce510 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:02.226132 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c68e3d-e552-48e4-a60c-d1a5052ce510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:02.226408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c68e3d-e552-48e4-a60c-d1a5052ce510 admin admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:02.227128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c68e3d-e552-48e4-a60c-d1a5052ce510 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:14:02.227440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c68e3d-e552-48e4-a60c-d1a5052ce510 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:02.229098 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c68e3d-e552-48e4-a60c-d1a5052ce510 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:02.240787 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1043/2074] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 05:14:02.245372 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c68e3d-e552-48e4-a60c-d1a5052ce510 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:02.255868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c68e3d-e552-48e4-a60c-d1a5052ce510 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:02.261221 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e10a3b1e-f21e-4cc2-9b4f-8b50959871ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '882275428a70495caeab73663243f476', 'request_id': 'req-e10a3b1e-f21e-4cc2-9b4f-8b50959871ba'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:02.264452 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:02.311666 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb44a29d-8a60-4a63-a34e-3e58ae49def1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:02.312272 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44a29d-8a60-4a63-a34e-3e58ae49def1 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:02.312552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44a29d-8a60-4a63-a34e-3e58ae49def1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:02.312826 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44a29d-8a60-4a63-a34e-3e58ae49def1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:02.313486 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb44a29d-8a60-4a63-a34e-3e58ae49def1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:02.316209 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb44a29d-8a60-4a63-a34e-3e58ae49def1 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:02.397024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e10a3b1e-f21e-4cc2-9b4f-8b50959871ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '882275428a70495caeab73663243f476', 'request_id': 'req-e10a3b1e-f21e-4cc2-9b4f-8b50959871ba'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:02.403221 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e10a3b1e-f21e-4cc2-9b4f-8b50959871ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '882275428a70495caeab73663243f476', 'request_id': 'req-e10a3b1e-f21e-4cc2-9b4f-8b50959871ba'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:02.414478 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e10a3b1e-f21e-4cc2-9b4f-8b50959871ba admin admin] The token cache is being invalidate because project 882275428a70495caeab73663243f476 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:14:02.414777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e10a3b1e-f21e-4cc2-9b4f-8b50959871ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:02.415455 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e10a3b1e-f21e-4cc2-9b4f-8b50959871ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:02.416971 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1032/2075] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:14:02 2022] DELETE /identity/v3/projects/882275428a70495caeab73663243f476 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:02.421661 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ebaf1-5d44-4406-9ffe-7c83e38ad3a5 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:02.421947 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ebaf1-5d44-4406-9ffe-7c83e38ad3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:02.422225 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ebaf1-5d44-4406-9ffe-7c83e38ad3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:02.442102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-756ebaf1-5d44-4406-9ffe-7c83e38ad3a5 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:02.443346 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-756ebaf1-5d44-4406-9ffe-7c83e38ad3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:02.445226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:02.451791 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1044/2076] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:14:02 2022] POST /identity/v3/projects => generated 371 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:14:02.451986 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1033/2077] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:14:02.456012 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb44a29d-8a60-4a63-a34e-3e58ae49def1 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:02.457835 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-bb44a29d-8a60-4a63-a34e-3e58ae49def1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:02.462632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ca9b2-dc09-4c2f-9f92-f5feb79b3c22 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:02.462953 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ca9b2-dc09-4c2f-9f92-f5feb79b3c22 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:02.463231 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ca9b2-dc09-4c2f-9f92-f5feb79b3c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:02.483612 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-c78ca9b2-dc09-4c2f-9f92-f5feb79b3c22 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:02.484820 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c78ca9b2-dc09-4c2f-9f92-f5feb79b3c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:02.522645 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb44a29d-8a60-4a63-a34e-3e58ae49def1 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:02.527222 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0698810-ae58-4c88-89fa-45e6a2128928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:02.527922 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0698810-ae58-4c88-89fa-45e6a2128928 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:02.528207 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0698810-ae58-4c88-89fa-45e6a2128928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:02.528522 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0698810-ae58-4c88-89fa-45e6a2128928 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:02.529195 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0698810-ae58-4c88-89fa-45e6a2128928 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:02.531154 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0698810-ae58-4c88-89fa-45e6a2128928 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:02.546562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1034/2078] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:02.551050 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78ca9b2-dc09-4c2f-9f92-f5feb79b3c22 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:02.560286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78ca9b2-dc09-4c2f-9f92-f5feb79b3c22 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:02.565775 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8bdfce-9c08-4e9c-998b-e5c7fa63cc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:02.566435 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bdfce-9c08-4e9c-998b-e5c7fa63cc1c admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:02.566719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bdfce-9c08-4e9c-998b-e5c7fa63cc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:02.567036 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bdfce-9c08-4e9c-998b-e5c7fa63cc1c admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:02.567725 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8bdfce-9c08-4e9c-998b-e5c7fa63cc1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:14:02.568033 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8bdfce-9c08-4e9c-998b-e5c7fa63cc1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:02.569627 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8bdfce-9c08-4e9c-998b-e5c7fa63cc1c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:02.582369 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1035/2079] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 05:14:02.586770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8bdfce-9c08-4e9c-998b-e5c7fa63cc1c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:02.596795 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8bdfce-9c08-4e9c-998b-e5c7fa63cc1c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:02.601906 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f98ad-d34a-403b-9c48-864ab010843e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:02.602497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f98ad-d34a-403b-9c48-864ab010843e admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:02.602785 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f98ad-d34a-403b-9c48-864ab010843e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:02.603109 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f98ad-d34a-403b-9c48-864ab010843e admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:02.603773 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674f98ad-d34a-403b-9c48-864ab010843e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:02.605660 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674f98ad-d34a-403b-9c48-864ab010843e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:02.641659 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1045/2080] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:14:02 2022] POST /identity/v3/users => generated 286 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:14:02.646381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0698810-ae58-4c88-89fa-45e6a2128928 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:02.658070 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0698810-ae58-4c88-89fa-45e6a2128928 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:02.666125 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788cc2c-11bf-49ef-bf45-d082760eaad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:02.666924 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788cc2c-11bf-49ef-bf45-d082760eaad9 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:02.667174 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788cc2c-11bf-49ef-bf45-d082760eaad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:02.667449 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788cc2c-11bf-49ef-bf45-d082760eaad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:02.668148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3788cc2c-11bf-49ef-bf45-d082760eaad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:02.669778 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3788cc2c-11bf-49ef-bf45-d082760eaad9 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:02.681241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1046/2081] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:14:02.691347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788cc2c-11bf-49ef-bf45-d082760eaad9 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:02.701528 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788cc2c-11bf-49ef-bf45-d082760eaad9 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:02.706776 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e72435-d27d-4c05-b4a5-f128f5ed0037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:02.707406 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e72435-d27d-4c05-b4a5-f128f5ed0037 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:02.707694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e72435-d27d-4c05-b4a5-f128f5ed0037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:02.707975 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e72435-d27d-4c05-b4a5-f128f5ed0037 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:02.708661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e72435-d27d-4c05-b4a5-f128f5ed0037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:02.710446 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e72435-d27d-4c05-b4a5-f128f5ed0037 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:02.726213 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1047/2082] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:14:02.727407 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e72435-d27d-4c05-b4a5-f128f5ed0037 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:02.738385 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e72435-d27d-4c05-b4a5-f128f5ed0037 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:02.743729 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a54b27-49d4-4920-b22a-010108e1c384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:02.745256 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a54b27-49d4-4920-b22a-010108e1c384 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:02.745256 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a54b27-49d4-4920-b22a-010108e1c384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:02.745256 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a54b27-49d4-4920-b22a-010108e1c384 admin admin] PATH_INFO: `/v3/projects/d9b28cfcd2da41528eb3d71199d3b2d9/users/8cbb45074cef4d5082580c4c5130eb9a/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:02.750399 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1036/2083] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:14:02 2022] POST /identity/v3/projects => generated 383 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:14:02.752046 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f98ad-d34a-403b-9c48-864ab010843e admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:02.767518 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a54b27-49d4-4920-b22a-010108e1c384 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9b28cfcd2da41528eb3d71199d3b2d9, user_id=8cbb45074cef4d5082580c4c5130eb9a, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:02.772608 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a54b27-49d4-4920-b22a-010108e1c384 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:02.824798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:02.845221 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f98ad-d34a-403b-9c48-864ab010843e admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:02.850512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69981fec-a0a3-4a43-b795-ec916f1e04c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:02.851178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69981fec-a0a3-4a43-b795-ec916f1e04c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:02.851465 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69981fec-a0a3-4a43-b795-ec916f1e04c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:02.851749 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69981fec-a0a3-4a43-b795-ec916f1e04c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:02.852421 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69981fec-a0a3-4a43-b795-ec916f1e04c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:02.854365 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69981fec-a0a3-4a43-b795-ec916f1e04c1 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:02.854978 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1048/2084] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:14:02 2022] PUT /identity/v3/projects/d9b28cfcd2da41528eb3d71199d3b2d9/users/8cbb45074cef4d5082580c4c5130eb9a/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:02.859990 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d1568-7dfb-4fc9-b865-e673711271af None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:02.860360 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d1568-7dfb-4fc9-b865-e673711271af None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:02.860944 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d1568-7dfb-4fc9-b865-e673711271af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:02.881120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-f35d1568-7dfb-4fc9-b865-e673711271af None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:02.882434 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-f35d1568-7dfb-4fc9-b865-e673711271af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:02.947626 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1049/2085] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:14:02.955989 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1037/2086] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:14:02 2022] POST /identity/v3/users => generated 292 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:02.959608 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d1568-7dfb-4fc9-b865-e673711271af None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:02.961652 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-f35d1568-7dfb-4fc9-b865-e673711271af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:02.961913 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69981fec-a0a3-4a43-b795-ec916f1e04c1 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:02.973411 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69981fec-a0a3-4a43-b795-ec916f1e04c1 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:02.980114 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ba78f-f877-4110-8b9b-6bb441fd118c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:02.980915 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ba78f-f877-4110-8b9b-6bb441fd118c admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:02.981230 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ba78f-f877-4110-8b9b-6bb441fd118c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:02.981894 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ba78f-f877-4110-8b9b-6bb441fd118c admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:02.982588 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33ba78f-f877-4110-8b9b-6bb441fd118c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:02.984408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33ba78f-f877-4110-8b9b-6bb441fd118c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:02.998987 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1038/2087] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:14:03.003130 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ba78f-f877-4110-8b9b-6bb441fd118c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:03.014366 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ba78f-f877-4110-8b9b-6bb441fd118c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:03.022902 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f575f0b-832a-4d54-90fe-2614d87e5ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:03.023587 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f575f0b-832a-4d54-90fe-2614d87e5ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:03.023876 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f575f0b-832a-4d54-90fe-2614d87e5ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:03.024159 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f575f0b-832a-4d54-90fe-2614d87e5ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:03.024875 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f575f0b-832a-4d54-90fe-2614d87e5ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:03.026732 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f575f0b-832a-4d54-90fe-2614d87e5ea7 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:03.035909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d1568-7dfb-4fc9-b865-e673711271af None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:03.040873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48ee28-6282-4e54-9bc8-aad79e4d298b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:03.041571 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48ee28-6282-4e54-9bc8-aad79e4d298b service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:03.041849 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48ee28-6282-4e54-9bc8-aad79e4d298b service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:03.042171 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48ee28-6282-4e54-9bc8-aad79e4d298b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:03.046102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1039/2088] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:14:03.051479 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f575f0b-832a-4d54-90fe-2614d87e5ea7 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:03.054703 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a48ee28-6282-4e54-9bc8-aad79e4d298b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:03.056221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a48ee28-6282-4e54-9bc8-aad79e4d298b service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:03.059398 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f575f0b-832a-4d54-90fe-2614d87e5ea7 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:03.066466 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33014293-c480-43f1-a9af-aca34b9b994a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:03.067535 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33014293-c480-43f1-a9af-aca34b9b994a admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:03.067829 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33014293-c480-43f1-a9af-aca34b9b994a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:03.068112 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33014293-c480-43f1-a9af-aca34b9b994a admin admin] PATH_INFO: `/v3/projects/0cf5f52a36cb479e96febcaf19896baf/users/279ec9aa1357413e91abaa062987aa27/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:03.072073 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1050/2089] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:14:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:14:03.094903 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33014293-c480-43f1-a9af-aca34b9b994a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cf5f52a36cb479e96febcaf19896baf, user_id=279ec9aa1357413e91abaa062987aa27, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:03.098271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33014293-c480-43f1-a9af-aca34b9b994a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:03.238622 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:03.246172 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1040/2090] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:14:03 2022] PUT /identity/v3/projects/0cf5f52a36cb479e96febcaf19896baf/users/279ec9aa1357413e91abaa062987aa27/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:14:03.258810 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1d67f-5f12-42e3-a7d2-ca23ff258dc1 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:03.259141 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1d67f-5f12-42e3-a7d2-ca23ff258dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:03.259417 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1d67f-5f12-42e3-a7d2-ca23ff258dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:03.302631 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-00d1d67f-5f12-42e3-a7d2-ca23ff258dc1 None None] MFA Rules not processed for user `279ec9aa1357413e91abaa062987aa27`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:03.303912 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-00d1d67f-5f12-42e3-a7d2-ca23ff258dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:03.405198 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1051/2091] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:03.418812 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33014293-c480-43f1-a9af-aca34b9b994a admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:03.420665 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-33014293-c480-43f1-a9af-aca34b9b994a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:03.487107 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33014293-c480-43f1-a9af-aca34b9b994a admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:03.490898 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4eb4bc-8ee9-4b18-8fbc-37feddf20a10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:03.491634 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4eb4bc-8ee9-4b18-8fbc-37feddf20a10 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:03.496678 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4eb4bc-8ee9-4b18-8fbc-37feddf20a10 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:03.496678 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4eb4bc-8ee9-4b18-8fbc-37feddf20a10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:03.502316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4eb4bc-8ee9-4b18-8fbc-37feddf20a10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:03.503915 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4eb4bc-8ee9-4b18-8fbc-37feddf20a10 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:03.513618 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1041/2092] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:14:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:14:03.671836 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:04.123460 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:04.607074 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddded1d2-b567-44ce-9f68-b7b200565a23 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:04.607366 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddded1d2-b567-44ce-9f68-b7b200565a23 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:04.607645 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddded1d2-b567-44ce-9f68-b7b200565a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:04.638555 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-ddded1d2-b567-44ce-9f68-b7b200565a23 None None] MFA Rules not processed for user `8cbb45074cef4d5082580c4c5130eb9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:04.639820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-ddded1d2-b567-44ce-9f68-b7b200565a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:04.713689 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1052/2093] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:04.718105 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:04.728901 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4eb4bc-8ee9-4b18-8fbc-37feddf20a10 service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:04.733890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4eb4bc-8ee9-4b18-8fbc-37feddf20a10 service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:04.737455 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106619bf-ab44-4aed-87e9-a4b7a2046afe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:04.738051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106619bf-ab44-4aed-87e9-a4b7a2046afe service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:04.738335 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106619bf-ab44-4aed-87e9-a4b7a2046afe service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:04.738608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106619bf-ab44-4aed-87e9-a4b7a2046afe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:04.747353 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106619bf-ab44-4aed-87e9-a4b7a2046afe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:04.751452 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106619bf-ab44-4aed-87e9-a4b7a2046afe service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:04.758369 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1042/2094] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:14:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:14:04.797524 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:05.196423 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:05.530654 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:06.414908 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:06.620978 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:06.626778 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66546d-a4cc-41c1-8718-68d99e6ae505 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:06.627139 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66546d-a4cc-41c1-8718-68d99e6ae505 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:06.627658 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66546d-a4cc-41c1-8718-68d99e6ae505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:06.659032 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-1c66546d-a4cc-41c1-8718-68d99e6ae505 None None] MFA Rules not processed for user `8cbb45074cef4d5082580c4c5130eb9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:06.660874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-1c66546d-a4cc-41c1-8718-68d99e6ae505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:06.673559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1053/2095] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:14:06.995661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:07.047131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:07.569741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:07.688636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:07.734327 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:08.020074 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:08.058178 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:08.363629 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:08.454796 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:08.831794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:11.879646 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106619bf-ab44-4aed-87e9-a4b7a2046afe service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:11.881459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-106619bf-ab44-4aed-87e9-a4b7a2046afe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:11.955340 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106619bf-ab44-4aed-87e9-a4b7a2046afe service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:11.959877 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b60710-980d-49d4-99a6-a7c279cc4110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:11.960563 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b60710-980d-49d4-99a6-a7c279cc4110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:11.960845 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b60710-980d-49d4-99a6-a7c279cc4110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:11.961165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b60710-980d-49d4-99a6-a7c279cc4110 admin admin] PATH_INFO: `/v3/users/41541854b74c49a886db2e7bfa867648` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:11.962615 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b60710-980d-49d4-99a6-a7c279cc4110 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41541854b74c49a886db2e7bfa867648)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:11.964704 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b60710-980d-49d4-99a6-a7c279cc4110 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:12.202464 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-39b60710-980d-49d4-99a6-a7c279cc4110 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41541854b74c49a886db2e7bfa867648'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:12.298087 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-39b60710-980d-49d4-99a6-a7c279cc4110 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41541854b74c49a886db2e7bfa867648'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:12.298865 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-39b60710-980d-49d4-99a6-a7c279cc4110 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41541854b74c49a886db2e7bfa867648'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:12.312032 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-39b60710-980d-49d4-99a6-a7c279cc4110 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41541854b74c49a886db2e7bfa867648'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:12.312724 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-39b60710-980d-49d4-99a6-a7c279cc4110 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41541854b74c49a886db2e7bfa867648'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:12.325758 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1043/2096] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:14:11 2022] DELETE /identity/v3/users/41541854b74c49a886db2e7bfa867648 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:14:12.890263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66546d-a4cc-41c1-8718-68d99e6ae505 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:12.904621 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66546d-a4cc-41c1-8718-68d99e6ae505 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:12.968833 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c7ca0-4ae8-4a57-a232-974603698319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:12.969563 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c7ca0-4ae8-4a57-a232-974603698319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:12.969827 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c7ca0-4ae8-4a57-a232-974603698319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:12.970109 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c7ca0-4ae8-4a57-a232-974603698319 admin admin] PATH_INFO: `/v3/projects/ec7ef948182a457381fd0a686ca298dc` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:12.971319 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c7ca0-4ae8-4a57-a232-974603698319 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec7ef948182a457381fd0a686ca298dc)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:12.973610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c7ca0-4ae8-4a57-a232-974603698319 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:13.078167 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-f83c7ca0-4ae8-4a57-a232-974603698319 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec7ef948182a457381fd0a686ca298dc', 'request_id': 'req-f83c7ca0-4ae8-4a57-a232-974603698319'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:13.084923 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-f83c7ca0-4ae8-4a57-a232-974603698319 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec7ef948182a457381fd0a686ca298dc', 'request_id': 'req-f83c7ca0-4ae8-4a57-a232-974603698319'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:13.220060 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-f83c7ca0-4ae8-4a57-a232-974603698319 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec7ef948182a457381fd0a686ca298dc', 'request_id': 'req-f83c7ca0-4ae8-4a57-a232-974603698319'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:13.231820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-f83c7ca0-4ae8-4a57-a232-974603698319 admin admin] The token cache is being invalidate because project ec7ef948182a457381fd0a686ca298dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:14:13.232113 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-f83c7ca0-4ae8-4a57-a232-974603698319 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:13.232758 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-f83c7ca0-4ae8-4a57-a232-974603698319 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:13.234550 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1054/2097] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:14:12 2022] DELETE /identity/v3/projects/ec7ef948182a457381fd0a686ca298dc => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:13.240049 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fd3fe-0a28-491a-8036-9fb3e9117666 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:13.240535 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fd3fe-0a28-491a-8036-9fb3e9117666 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:13.240646 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fd3fe-0a28-491a-8036-9fb3e9117666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:13.260133 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-7f0fd3fe-0a28-491a-8036-9fb3e9117666 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:13.261321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7f0fd3fe-0a28-491a-8036-9fb3e9117666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:13.309966 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:13.322592 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1044/2098] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:14:13.330656 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12918101-1366-4bf5-88a7-523e64ab61f5 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:13.330988 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12918101-1366-4bf5-88a7-523e64ab61f5 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:13.331272 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12918101-1366-4bf5-88a7-523e64ab61f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:13.351686 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-12918101-1366-4bf5-88a7-523e64ab61f5 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:13.352910 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-12918101-1366-4bf5-88a7-523e64ab61f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:13.361471 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1055/2099] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:13.365854 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fd3fe-0a28-491a-8036-9fb3e9117666 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:13.375653 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fd3fe-0a28-491a-8036-9fb3e9117666 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:13.380381 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9728b534-d727-4e7f-a48b-178b5e29a147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:13.381029 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728b534-d727-4e7f-a48b-178b5e29a147 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:13.381312 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728b534-d727-4e7f-a48b-178b5e29a147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:13.385364 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728b534-d727-4e7f-a48b-178b5e29a147 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:13.385364 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9728b534-d727-4e7f-a48b-178b5e29a147 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:14:13.385364 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9728b534-d727-4e7f-a48b-178b5e29a147 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:13.385364 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9728b534-d727-4e7f-a48b-178b5e29a147 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:13.399620 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1045/2100] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:14:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 05:14:13.401400 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12918101-1366-4bf5-88a7-523e64ab61f5 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:13.411568 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12918101-1366-4bf5-88a7-523e64ab61f5 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:13.416505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcbe924-703b-4cb8-ab8a-7f17b1db7ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:13.417128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcbe924-703b-4cb8-ab8a-7f17b1db7ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:13.417417 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcbe924-703b-4cb8-ab8a-7f17b1db7ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:13.417697 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcbe924-703b-4cb8-ab8a-7f17b1db7ec7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:13.418368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcbe924-703b-4cb8-ab8a-7f17b1db7ec7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:13.420331 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcbe924-703b-4cb8-ab8a-7f17b1db7ec7 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:13.437606 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:13.510889 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1056/2101] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:14:13 2022] POST /identity/v3/projects => generated 399 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:13.515055 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9728b534-d727-4e7f-a48b-178b5e29a147 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:13.525699 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9728b534-d727-4e7f-a48b-178b5e29a147 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:13.584760 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd4127-feb8-4c16-87e8-ab7f654984ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:13.585403 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd4127-feb8-4c16-87e8-ab7f654984ef admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:13.585694 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd4127-feb8-4c16-87e8-ab7f654984ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:13.585972 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd4127-feb8-4c16-87e8-ab7f654984ef admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:13.586649 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd4127-feb8-4c16-87e8-ab7f654984ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:13.588627 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd4127-feb8-4c16-87e8-ab7f654984ef admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:13.749831 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1046/2102] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:14:13 2022] POST /identity/v3/users => generated 300 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:13.754643 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcbe924-703b-4cb8-ab8a-7f17b1db7ec7 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:13.765482 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcbe924-703b-4cb8-ab8a-7f17b1db7ec7 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:13.770264 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e2fa7-1e74-4064-aa0f-8f47c8141d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:13.770911 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e2fa7-1e74-4064-aa0f-8f47c8141d61 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:13.771196 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e2fa7-1e74-4064-aa0f-8f47c8141d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:13.771471 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e2fa7-1e74-4064-aa0f-8f47c8141d61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:13.772200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3e2fa7-1e74-4064-aa0f-8f47c8141d61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:13.773881 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3e2fa7-1e74-4064-aa0f-8f47c8141d61 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:13.784983 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1057/2103] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:14:13.789660 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd4127-feb8-4c16-87e8-ab7f654984ef admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:13.800868 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd4127-feb8-4c16-87e8-ab7f654984ef admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:13.806273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3116002c-49cc-46c2-9ad0-139fe62acc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:13.807176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116002c-49cc-46c2-9ad0-139fe62acc6b admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:13.807296 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116002c-49cc-46c2-9ad0-139fe62acc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:13.807483 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116002c-49cc-46c2-9ad0-139fe62acc6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:13.808186 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3116002c-49cc-46c2-9ad0-139fe62acc6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:13.809795 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3116002c-49cc-46c2-9ad0-139fe62acc6b admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:13.821902 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1047/2104] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:14:13.826470 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e2fa7-1e74-4064-aa0f-8f47c8141d61 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:13.836748 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e2fa7-1e74-4064-aa0f-8f47c8141d61 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:13.841284 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27e345-4523-437f-8486-59ddfcfcff91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:13.841937 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27e345-4523-437f-8486-59ddfcfcff91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:13.842220 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27e345-4523-437f-8486-59ddfcfcff91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:13.842499 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27e345-4523-437f-8486-59ddfcfcff91 admin admin] PATH_INFO: `/v3/projects/bc8e3c578de04aab9bc406861ffab4a7/users/f42e668f5a1a499a9d16c0fba456449a/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:13.857652 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d27e345-4523-437f-8486-59ddfcfcff91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc8e3c578de04aab9bc406861ffab4a7, user_id=f42e668f5a1a499a9d16c0fba456449a, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:13.860937 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d27e345-4523-437f-8486-59ddfcfcff91 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:13.938528 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1058/2105] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:14:13 2022] PUT /identity/v3/projects/bc8e3c578de04aab9bc406861ffab4a7/users/f42e668f5a1a499a9d16c0fba456449a/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:13.948878 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00574a-e710-4dfb-a66d-c2355d2327db None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:13.949181 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00574a-e710-4dfb-a66d-c2355d2327db None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:13.953784 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00574a-e710-4dfb-a66d-c2355d2327db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:13.979526 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-1c00574a-e710-4dfb-a66d-c2355d2327db None None] MFA Rules not processed for user `f42e668f5a1a499a9d16c0fba456449a`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:13.980786 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1c00574a-e710-4dfb-a66d-c2355d2327db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:14.045698 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1048/2106] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:13 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:14.050891 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27e345-4523-437f-8486-59ddfcfcff91 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:14.118725 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27e345-4523-437f-8486-59ddfcfcff91 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:14.123636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1776ad-418d-4ecf-a01c-ab4cf07778ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:14.124250 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1776ad-418d-4ecf-a01c-ab4cf07778ea admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:14.124538 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1776ad-418d-4ecf-a01c-ab4cf07778ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:14.124818 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1776ad-418d-4ecf-a01c-ab4cf07778ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:14.125519 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1776ad-418d-4ecf-a01c-ab4cf07778ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:14.127497 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1776ad-418d-4ecf-a01c-ab4cf07778ea admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:14.239195 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1059/2107] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:14:14 2022] POST /identity/v3/projects => generated 399 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:14:14.243370 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00574a-e710-4dfb-a66d-c2355d2327db None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:14.254410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00574a-e710-4dfb-a66d-c2355d2327db None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:14.302966 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:14.312947 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375cfce-9c14-429e-9a98-f09e0e4c3fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:14.313623 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375cfce-9c14-429e-9a98-f09e0e4c3fc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:14.313926 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375cfce-9c14-429e-9a98-f09e0e4c3fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:14.314191 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375cfce-9c14-429e-9a98-f09e0e4c3fc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:14.314917 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4375cfce-9c14-429e-9a98-f09e0e4c3fc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:14.316884 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4375cfce-9c14-429e-9a98-f09e0e4c3fc4 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:14.427651 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1049/2108] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:14:14 2022] POST /identity/v3/users => generated 306 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:14.432801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1776ad-418d-4ecf-a01c-ab4cf07778ea admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:14.443357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1776ad-418d-4ecf-a01c-ab4cf07778ea admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:14.447939 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b5714-7b32-4896-acf2-bbbc6e438200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:14.448548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b5714-7b32-4896-acf2-bbbc6e438200 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:14.448826 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b5714-7b32-4896-acf2-bbbc6e438200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:14.449107 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b5714-7b32-4896-acf2-bbbc6e438200 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:14.449826 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3b5714-7b32-4896-acf2-bbbc6e438200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:14.451465 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3b5714-7b32-4896-acf2-bbbc6e438200 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:14.463775 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1060/2109] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:14:14.468070 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375cfce-9c14-429e-9a98-f09e0e4c3fc4 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:14.482284 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375cfce-9c14-429e-9a98-f09e0e4c3fc4 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:14.487722 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87fa7c6-1b72-4a8d-8263-11d4a13b2e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:14.488501 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87fa7c6-1b72-4a8d-8263-11d4a13b2e9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:14.488857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87fa7c6-1b72-4a8d-8263-11d4a13b2e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:14.489207 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87fa7c6-1b72-4a8d-8263-11d4a13b2e9e admin admin] PATH_INFO: `/v3/projects/b5e409d5fe204d2cb86fab77ca60ab41/users/228c56bccacb4ddca847f799fc24dd12/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:14.505024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87fa7c6-1b72-4a8d-8263-11d4a13b2e9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5e409d5fe204d2cb86fab77ca60ab41, user_id=228c56bccacb4ddca847f799fc24dd12, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:14.507889 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87fa7c6-1b72-4a8d-8263-11d4a13b2e9e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:14.605477 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1050/2110] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:14:14 2022] PUT /identity/v3/projects/b5e409d5fe204d2cb86fab77ca60ab41/users/228c56bccacb4ddca847f799fc24dd12/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:14:14.609308 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b5714-7b32-4896-acf2-bbbc6e438200 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:14.620186 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b5714-7b32-4896-acf2-bbbc6e438200 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:14.723868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccae157d-bea7-473a-8428-8b65eeac2313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:14.724612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae157d-bea7-473a-8428-8b65eeac2313 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:14.725157 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae157d-bea7-473a-8428-8b65eeac2313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:14.725507 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae157d-bea7-473a-8428-8b65eeac2313 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:14.726343 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccae157d-bea7-473a-8428-8b65eeac2313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:14.728106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccae157d-bea7-473a-8428-8b65eeac2313 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:14.739410 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1061/2111] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:14:14.743520 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87fa7c6-1b72-4a8d-8263-11d4a13b2e9e admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:14.755096 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87fa7c6-1b72-4a8d-8263-11d4a13b2e9e admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:14.759893 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85358a-e64a-4d81-a2c6-b648dc6ee91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:14.760565 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85358a-e64a-4d81-a2c6-b648dc6ee91e admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:14.760853 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85358a-e64a-4d81-a2c6-b648dc6ee91e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:14.761134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85358a-e64a-4d81-a2c6-b648dc6ee91e admin admin] PATH_INFO: `/v3/projects/b5e409d5fe204d2cb86fab77ca60ab41/users/228c56bccacb4ddca847f799fc24dd12/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:14.763110 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e85358a-e64a-4d81-a2c6-b648dc6ee91e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5e409d5fe204d2cb86fab77ca60ab41, user_id=228c56bccacb4ddca847f799fc24dd12, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:14.766031 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e85358a-e64a-4d81-a2c6-b648dc6ee91e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:14.776864 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1051/2112] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:14:14 2022] PUT /identity/v3/projects/b5e409d5fe204d2cb86fab77ca60ab41/users/228c56bccacb4ddca847f799fc24dd12/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:14.788521 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfaf31d-406b-45e0-9797-dd14d1f5c847 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:14.788811 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfaf31d-406b-45e0-9797-dd14d1f5c847 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:14.789107 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfaf31d-406b-45e0-9797-dd14d1f5c847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:14.818071 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-ccfaf31d-406b-45e0-9797-dd14d1f5c847 None None] MFA Rules not processed for user `228c56bccacb4ddca847f799fc24dd12`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:14.819348 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-ccfaf31d-406b-45e0-9797-dd14d1f5c847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:14.839535 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:14.900781 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1062/2113] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:14 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:14.908932 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85358a-e64a-4d81-a2c6-b648dc6ee91e admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:14.910882 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8e85358a-e64a-4d81-a2c6-b648dc6ee91e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:14.980133 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85358a-e64a-4d81-a2c6-b648dc6ee91e admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:14.998300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c84a3-2574-4051-a9e5-bf4c7e10eb2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:14.998300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c84a3-2574-4051-a9e5-bf4c7e10eb2d service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:14.998300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c84a3-2574-4051-a9e5-bf4c7e10eb2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:14.998618 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c84a3-2574-4051-a9e5-bf4c7e10eb2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:15.015165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c84a3-2574-4051-a9e5-bf4c7e10eb2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:15.017298 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c84a3-2574-4051-a9e5-bf4c7e10eb2d service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:15.050765 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1052/2114] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:14:14 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:14:15.340688 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfaf31d-406b-45e0-9797-dd14d1f5c847 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:15.356517 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfaf31d-406b-45e0-9797-dd14d1f5c847 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:15.368553 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cda67b-82c7-47ad-b380-dc14d99a9400 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:15.368553 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cda67b-82c7-47ad-b380-dc14d99a9400 service cinder] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:15.368553 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cda67b-82c7-47ad-b380-dc14d99a9400 service cinder] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:15.368553 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cda67b-82c7-47ad-b380-dc14d99a9400 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:15.434785 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cda67b-82c7-47ad-b380-dc14d99a9400 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:15.436293 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cda67b-82c7-47ad-b380-dc14d99a9400 service cinder] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:15.450176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1063/2115] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:14:15 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:14:17.083537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c84a3-2574-4051-a9e5-bf4c7e10eb2d service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:17.085564 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ab5c84a3-2574-4051-a9e5-bf4c7e10eb2d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:17.241599 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c84a3-2574-4051-a9e5-bf4c7e10eb2d service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:17.246716 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6910e70-fcaf-45b6-9cc4-694853e637f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:17.247434 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6910e70-fcaf-45b6-9cc4-694853e637f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:17.247690 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6910e70-fcaf-45b6-9cc4-694853e637f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:17.247969 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6910e70-fcaf-45b6-9cc4-694853e637f1 admin admin] PATH_INFO: `/v3/users/279ec9aa1357413e91abaa062987aa27` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:17.250266 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6910e70-fcaf-45b6-9cc4-694853e637f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=279ec9aa1357413e91abaa062987aa27)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:17.252451 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6910e70-fcaf-45b6-9cc4-694853e637f1 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:17.475238 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d6910e70-fcaf-45b6-9cc4-694853e637f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '279ec9aa1357413e91abaa062987aa27'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:17.476814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:17.537822 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d6910e70-fcaf-45b6-9cc4-694853e637f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '279ec9aa1357413e91abaa062987aa27'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:17.538465 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d6910e70-fcaf-45b6-9cc4-694853e637f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '279ec9aa1357413e91abaa062987aa27'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:17.559249 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d6910e70-fcaf-45b6-9cc4-694853e637f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '279ec9aa1357413e91abaa062987aa27'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:17.560025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d6910e70-fcaf-45b6-9cc4-694853e637f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '279ec9aa1357413e91abaa062987aa27'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:17.573756 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1053/2116] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:14:17 2022] DELETE /identity/v3/users/279ec9aa1357413e91abaa062987aa27 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:17.580960 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb0f82-c314-493f-b0c5-a74d5ad03814 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:17.581436 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb0f82-c314-493f-b0c5-a74d5ad03814 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:17.581529 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb0f82-c314-493f-b0c5-a74d5ad03814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:17.600721 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-1bcb0f82-c314-493f-b0c5-a74d5ad03814 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:17.601928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-1bcb0f82-c314-493f-b0c5-a74d5ad03814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:17.672438 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1064/2117] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:17.678604 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6910e70-fcaf-45b6-9cc4-694853e637f1 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:17.680058 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d6910e70-fcaf-45b6-9cc4-694853e637f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:17.738230 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6910e70-fcaf-45b6-9cc4-694853e637f1 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:17.741920 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e938b9-3aa8-4251-bfd4-76bc02c006c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:17.742548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e938b9-3aa8-4251-bfd4-76bc02c006c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:17.742832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e938b9-3aa8-4251-bfd4-76bc02c006c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:17.743140 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e938b9-3aa8-4251-bfd4-76bc02c006c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:17.753728 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e938b9-3aa8-4251-bfd4-76bc02c006c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:17.755337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e938b9-3aa8-4251-bfd4-76bc02c006c4 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:17.769339 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1054/2118] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:14:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:14:18.210670 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb0f82-c314-493f-b0c5-a74d5ad03814 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:18.212415 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-1bcb0f82-c314-493f-b0c5-a74d5ad03814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:18.227517 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb0f82-c314-493f-b0c5-a74d5ad03814 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:18.235532 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3222f0-2fca-48de-aaf7-58ab05183077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:18.235532 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3222f0-2fca-48de-aaf7-58ab05183077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:18.235532 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3222f0-2fca-48de-aaf7-58ab05183077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:18.235532 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3222f0-2fca-48de-aaf7-58ab05183077 admin admin] PATH_INFO: `/v3/projects/0cf5f52a36cb479e96febcaf19896baf` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:18.236649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3222f0-2fca-48de-aaf7-58ab05183077 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cf5f52a36cb479e96febcaf19896baf)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:18.238739 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3222f0-2fca-48de-aaf7-58ab05183077 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:18.366446 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-be3222f0-2fca-48de-aaf7-58ab05183077 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cf5f52a36cb479e96febcaf19896baf', 'request_id': 'req-be3222f0-2fca-48de-aaf7-58ab05183077'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:18.367696 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:18.373560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-be3222f0-2fca-48de-aaf7-58ab05183077 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cf5f52a36cb479e96febcaf19896baf', 'request_id': 'req-be3222f0-2fca-48de-aaf7-58ab05183077'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:18.466377 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-be3222f0-2fca-48de-aaf7-58ab05183077 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cf5f52a36cb479e96febcaf19896baf', 'request_id': 'req-be3222f0-2fca-48de-aaf7-58ab05183077'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:18.485200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-be3222f0-2fca-48de-aaf7-58ab05183077 admin admin] The token cache is being invalidate because project 0cf5f52a36cb479e96febcaf19896baf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:14:18.485621 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-be3222f0-2fca-48de-aaf7-58ab05183077 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:18.486697 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-be3222f0-2fca-48de-aaf7-58ab05183077 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:18.489398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1065/2119] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:14:18 2022] DELETE /identity/v3/projects/0cf5f52a36cb479e96febcaf19896baf => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:18.494303 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e3027-da89-426a-be97-b92b5a388739 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:18.494779 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e3027-da89-426a-be97-b92b5a388739 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:18.494901 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e3027-da89-426a-be97-b92b5a388739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:18.517171 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-4a7e3027-da89-426a-be97-b92b5a388739 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:18.518497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-4a7e3027-da89-426a-be97-b92b5a388739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:18.584626 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1055/2120] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:18.594536 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0420e-1ac2-4deb-9eaf-4ec5af7938a1 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:18.594536 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0420e-1ac2-4deb-9eaf-4ec5af7938a1 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:18.594536 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0420e-1ac2-4deb-9eaf-4ec5af7938a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:18.613856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-baf0420e-1ac2-4deb-9eaf-4ec5af7938a1 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:18.615117 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-baf0420e-1ac2-4deb-9eaf-4ec5af7938a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:18.625027 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1066/2121] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:18.634448 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e3027-da89-426a-be97-b92b5a388739 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:18.644176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e3027-da89-426a-be97-b92b5a388739 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:18.649845 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a97ff-cda2-44e5-91ec-6e8249720c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:18.650473 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a97ff-cda2-44e5-91ec-6e8249720c66 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:18.650759 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a97ff-cda2-44e5-91ec-6e8249720c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:18.651069 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a97ff-cda2-44e5-91ec-6e8249720c66 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:18.651774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a97ff-cda2-44e5-91ec-6e8249720c66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:14:18.652085 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a97ff-cda2-44e5-91ec-6e8249720c66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:18.653755 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a97ff-cda2-44e5-91ec-6e8249720c66 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:18.669865 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1056/2122] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:14:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:14:18.674182 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0420e-1ac2-4deb-9eaf-4ec5af7938a1 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:18.683566 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0420e-1ac2-4deb-9eaf-4ec5af7938a1 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:18.689811 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0337559-b981-4a9c-8309-14b9ca5df8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:18.690431 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0337559-b981-4a9c-8309-14b9ca5df8e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:18.690718 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0337559-b981-4a9c-8309-14b9ca5df8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:18.691068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0337559-b981-4a9c-8309-14b9ca5df8e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:18.691740 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0337559-b981-4a9c-8309-14b9ca5df8e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:18.693661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0337559-b981-4a9c-8309-14b9ca5df8e0 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:18.779616 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1067/2123] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:14:18 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:14:18.784410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a97ff-cda2-44e5-91ec-6e8249720c66 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:18.794770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a97ff-cda2-44e5-91ec-6e8249720c66 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:18.848512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:18.855276 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb6caa-517d-48c2-b8ad-accba3f9222c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:18.855894 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb6caa-517d-48c2-b8ad-accba3f9222c admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:18.856176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb6caa-517d-48c2-b8ad-accba3f9222c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:18.856450 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb6caa-517d-48c2-b8ad-accba3f9222c admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:18.857118 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb6caa-517d-48c2-b8ad-accba3f9222c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:18.859161 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb6caa-517d-48c2-b8ad-accba3f9222c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:18.955881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1057/2124] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:14:18 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:18.960088 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0337559-b981-4a9c-8309-14b9ca5df8e0 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:18.981298 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0337559-b981-4a9c-8309-14b9ca5df8e0 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:18.981298 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccedecd-505c-4d53-9316-0f77b7316e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:18.981298 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccedecd-505c-4d53-9316-0f77b7316e1d admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:18.981298 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccedecd-505c-4d53-9316-0f77b7316e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:18.981736 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccedecd-505c-4d53-9316-0f77b7316e1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:18.981736 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccedecd-505c-4d53-9316-0f77b7316e1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:18.981736 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccedecd-505c-4d53-9316-0f77b7316e1d admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:18.994771 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1068/2125] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:14:19.006023 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb6caa-517d-48c2-b8ad-accba3f9222c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:19.018955 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb6caa-517d-48c2-b8ad-accba3f9222c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:19.024777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2e93d-20d9-4b3b-9fd2-773ec56690b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:19.025426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2e93d-20d9-4b3b-9fd2-773ec56690b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:19.025712 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2e93d-20d9-4b3b-9fd2-773ec56690b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:19.025987 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2e93d-20d9-4b3b-9fd2-773ec56690b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:19.026690 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2e93d-20d9-4b3b-9fd2-773ec56690b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:19.028365 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2e93d-20d9-4b3b-9fd2-773ec56690b4 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:19.041565 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1058/2126] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:14:19.044482 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccedecd-505c-4d53-9316-0f77b7316e1d admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:19.055449 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccedecd-505c-4d53-9316-0f77b7316e1d admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:19.060450 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4838c-c2c2-4738-9077-0d61d0976747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:19.061116 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4838c-c2c2-4738-9077-0d61d0976747 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:19.064408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4838c-c2c2-4738-9077-0d61d0976747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:19.064408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4838c-c2c2-4738-9077-0d61d0976747 admin admin] PATH_INFO: `/v3/projects/5768143089fe4d1d9cffd180c16b88a1/users/3cc20dc10ec74ae1bd814b183991e3a2/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:19.082650 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4838c-c2c2-4738-9077-0d61d0976747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5768143089fe4d1d9cffd180c16b88a1, user_id=3cc20dc10ec74ae1bd814b183991e3a2, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:19.085639 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4838c-c2c2-4738-9077-0d61d0976747 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:19.173878 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1069/2127] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:14:19 2022] PUT /identity/v3/projects/5768143089fe4d1d9cffd180c16b88a1/users/3cc20dc10ec74ae1bd814b183991e3a2/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:19.184306 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e6d68-385e-4f55-b681-e2d5d649c4a1 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:19.184608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e6d68-385e-4f55-b681-e2d5d649c4a1 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:19.184951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e6d68-385e-4f55-b681-e2d5d649c4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:19.219912 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-fc7e6d68-385e-4f55-b681-e2d5d649c4a1 None None] MFA Rules not processed for user `3cc20dc10ec74ae1bd814b183991e3a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:19.221130 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-fc7e6d68-385e-4f55-b681-e2d5d649c4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:19.289294 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1059/2128] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:19.332913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4838c-c2c2-4738-9077-0d61d0976747 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:19.334788 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-4eb4838c-c2c2-4738-9077-0d61d0976747 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:19.399518 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4838c-c2c2-4738-9077-0d61d0976747 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:19.403398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46da47a-a3d2-46e3-9804-44bc000edb91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:19.404033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46da47a-a3d2-46e3-9804-44bc000edb91 service cinder] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:19.404521 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46da47a-a3d2-46e3-9804-44bc000edb91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:19.404765 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46da47a-a3d2-46e3-9804-44bc000edb91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:19.415703 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46da47a-a3d2-46e3-9804-44bc000edb91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:19.417211 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46da47a-a3d2-46e3-9804-44bc000edb91 service cinder] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:19.427154 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1070/2129] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:14:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:14:20.163719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e6d68-385e-4f55-b681-e2d5d649c4a1 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:20.165207 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-fc7e6d68-385e-4f55-b681-e2d5d649c4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:20.238987 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e6d68-385e-4f55-b681-e2d5d649c4a1 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:20.243798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8a8c25-6dbc-4874-ae05-ec4855639cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:20.244456 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a8c25-6dbc-4874-ae05-ec4855639cb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:20.244739 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a8c25-6dbc-4874-ae05-ec4855639cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:20.245044 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a8c25-6dbc-4874-ae05-ec4855639cb8 admin admin] PATH_INFO: `/v3/users/27eaa7e3daee4b4587d8dad39bf4a42a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:20.246251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8a8c25-6dbc-4874-ae05-ec4855639cb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27eaa7e3daee4b4587d8dad39bf4a42a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:20.248262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8a8c25-6dbc-4874-ae05-ec4855639cb8 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:20.414354 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-dc8a8c25-6dbc-4874-ae05-ec4855639cb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27eaa7e3daee4b4587d8dad39bf4a42a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:20.483517 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-dc8a8c25-6dbc-4874-ae05-ec4855639cb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27eaa7e3daee4b4587d8dad39bf4a42a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:20.484315 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-dc8a8c25-6dbc-4874-ae05-ec4855639cb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27eaa7e3daee4b4587d8dad39bf4a42a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:20.494176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:20.499253 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-dc8a8c25-6dbc-4874-ae05-ec4855639cb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27eaa7e3daee4b4587d8dad39bf4a42a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:20.499963 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-dc8a8c25-6dbc-4874-ae05-ec4855639cb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27eaa7e3daee4b4587d8dad39bf4a42a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:20.514502 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1060/2130] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:14:20 2022] DELETE /identity/v3/users/27eaa7e3daee4b4587d8dad39bf4a42a => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:14:20.523244 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa083d58-fb80-4ef9-907e-e6e2a7c89da4 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:20.523244 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa083d58-fb80-4ef9-907e-e6e2a7c89da4 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:20.523244 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa083d58-fb80-4ef9-907e-e6e2a7c89da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:20.543146 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-fa083d58-fb80-4ef9-907e-e6e2a7c89da4 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:20.543506 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-fa083d58-fb80-4ef9-907e-e6e2a7c89da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:20.623677 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1071/2131] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:20.634473 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8a8c25-6dbc-4874-ae05-ec4855639cb8 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:20.635911 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-dc8a8c25-6dbc-4874-ae05-ec4855639cb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:20.725755 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8a8c25-6dbc-4874-ae05-ec4855639cb8 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:20.730203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a80fc6-1cf9-4de7-b486-8e3d159f0c2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:20.731415 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a80fc6-1cf9-4de7-b486-8e3d159f0c2a service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:20.731415 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a80fc6-1cf9-4de7-b486-8e3d159f0c2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:20.731859 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a80fc6-1cf9-4de7-b486-8e3d159f0c2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:20.744838 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a80fc6-1cf9-4de7-b486-8e3d159f0c2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:20.744838 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a80fc6-1cf9-4de7-b486-8e3d159f0c2a service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:20.745953 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa083d58-fb80-4ef9-907e-e6e2a7c89da4 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:20.748850 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-fa083d58-fb80-4ef9-907e-e6e2a7c89da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:20.757274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1061/2132] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:14:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:14:20.774357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa083d58-fb80-4ef9-907e-e6e2a7c89da4 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:20.779570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555bca1-5177-424a-9a4a-576a63d1d166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:20.780340 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555bca1-5177-424a-9a4a-576a63d1d166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:20.780632 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555bca1-5177-424a-9a4a-576a63d1d166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:20.780919 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555bca1-5177-424a-9a4a-576a63d1d166 admin admin] PATH_INFO: `/v3/users/f42e668f5a1a499a9d16c0fba456449a` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:20.782222 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1555bca1-5177-424a-9a4a-576a63d1d166 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f42e668f5a1a499a9d16c0fba456449a)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:20.784374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1555bca1-5177-424a-9a4a-576a63d1d166 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:20.804799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:21.079413 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-1555bca1-5177-424a-9a4a-576a63d1d166 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f42e668f5a1a499a9d16c0fba456449a'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:21.173865 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-1555bca1-5177-424a-9a4a-576a63d1d166 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f42e668f5a1a499a9d16c0fba456449a'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:21.174631 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-1555bca1-5177-424a-9a4a-576a63d1d166 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f42e668f5a1a499a9d16c0fba456449a'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:21.191513 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-1555bca1-5177-424a-9a4a-576a63d1d166 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f42e668f5a1a499a9d16c0fba456449a'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:21.192454 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-1555bca1-5177-424a-9a4a-576a63d1d166 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f42e668f5a1a499a9d16c0fba456449a'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:21.222552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1072/2133] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:14:20 2022] DELETE /identity/v3/users/f42e668f5a1a499a9d16c0fba456449a => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:21.222872 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2ac2f-fa26-42ed-8c9a-8366212a7e5c None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:21.222872 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2ac2f-fa26-42ed-8c9a-8366212a7e5c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:21.223677 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2ac2f-fa26-42ed-8c9a-8366212a7e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:21.247564 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-e8f2ac2f-fa26-42ed-8c9a-8366212a7e5c None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:21.249279 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e8f2ac2f-fa26-42ed-8c9a-8366212a7e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:21.316223 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1062/2134] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:21.325799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555bca1-5177-424a-9a4a-576a63d1d166 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:21.331313 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-1555bca1-5177-424a-9a4a-576a63d1d166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:21.386875 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555bca1-5177-424a-9a4a-576a63d1d166 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:21.390739 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05abd1a-d849-4480-bbfc-aa556f303556 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:21.392696 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05abd1a-d849-4480-bbfc-aa556f303556 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:21.392696 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05abd1a-d849-4480-bbfc-aa556f303556 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:21.392696 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05abd1a-d849-4480-bbfc-aa556f303556 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:21.407508 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05abd1a-d849-4480-bbfc-aa556f303556 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:21.408986 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05abd1a-d849-4480-bbfc-aa556f303556 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:21.420739 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2ac2f-fa26-42ed-8c9a-8366212a7e5c None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:21.423678 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e8f2ac2f-fa26-42ed-8c9a-8366212a7e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:21.443793 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2ac2f-fa26-42ed-8c9a-8366212a7e5c None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:21.445398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1073/2135] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:14:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:14:21.459271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17fb51-95d0-4c60-9b38-260688421b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:21.460066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17fb51-95d0-4c60-9b38-260688421b77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:21.460463 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17fb51-95d0-4c60-9b38-260688421b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:21.460826 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17fb51-95d0-4c60-9b38-260688421b77 admin admin] PATH_INFO: `/v3/projects/7b2aac05d35042f6b7e0603d59f7385f` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:21.462090 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f17fb51-95d0-4c60-9b38-260688421b77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b2aac05d35042f6b7e0603d59f7385f)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:21.464339 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f17fb51-95d0-4c60-9b38-260688421b77 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:21.566011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0f17fb51-95d0-4c60-9b38-260688421b77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b2aac05d35042f6b7e0603d59f7385f', 'request_id': 'req-0f17fb51-95d0-4c60-9b38-260688421b77'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:21.654427 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0f17fb51-95d0-4c60-9b38-260688421b77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b2aac05d35042f6b7e0603d59f7385f', 'request_id': 'req-0f17fb51-95d0-4c60-9b38-260688421b77'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:21.669217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0f17fb51-95d0-4c60-9b38-260688421b77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b2aac05d35042f6b7e0603d59f7385f', 'request_id': 'req-0f17fb51-95d0-4c60-9b38-260688421b77'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:21.681997 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0f17fb51-95d0-4c60-9b38-260688421b77 admin admin] The token cache is being invalidate because project 7b2aac05d35042f6b7e0603d59f7385f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:14:21.682406 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0f17fb51-95d0-4c60-9b38-260688421b77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:21.683005 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0f17fb51-95d0-4c60-9b38-260688421b77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:21.684831 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1063/2136] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:14:21 2022] DELETE /identity/v3/projects/7b2aac05d35042f6b7e0603d59f7385f => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:14:21.690088 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a98fc-da9d-4dd4-863b-24195fc9feee None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:21.690579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a98fc-da9d-4dd4-863b-24195fc9feee None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:21.690682 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a98fc-da9d-4dd4-863b-24195fc9feee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:21.712203 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-b80a98fc-da9d-4dd4-863b-24195fc9feee None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:21.713633 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-b80a98fc-da9d-4dd4-863b-24195fc9feee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:21.788797 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1074/2137] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:21.797857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2545a-8e9e-40c7-8e27-615a571d96e1 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:21.797857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2545a-8e9e-40c7-8e27-615a571d96e1 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:21.797857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2545a-8e9e-40c7-8e27-615a571d96e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:21.822261 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-79c2545a-8e9e-40c7-8e27-615a571d96e1 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:21.829506 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-79c2545a-8e9e-40c7-8e27-615a571d96e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:21.832727 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1064/2138] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:14:21.836537 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a98fc-da9d-4dd4-863b-24195fc9feee None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:21.848188 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a98fc-da9d-4dd4-863b-24195fc9feee None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:21.856497 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce72176-996d-4823-bc62-773e093a65d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:21.857228 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce72176-996d-4823-bc62-773e093a65d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:21.857579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce72176-996d-4823-bc62-773e093a65d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:21.857960 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce72176-996d-4823-bc62-773e093a65d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:21.858759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce72176-996d-4823-bc62-773e093a65d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:14:21.859158 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce72176-996d-4823-bc62-773e093a65d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:21.862786 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce72176-996d-4823-bc62-773e093a65d2 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:21.879386 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1075/2139] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:14:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:14:21.886825 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2545a-8e9e-40c7-8e27-615a571d96e1 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:21.891939 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:21.897186 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2545a-8e9e-40c7-8e27-615a571d96e1 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:21.902304 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785b528-ee1b-4615-8a84-4c75863056fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:21.903136 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785b528-ee1b-4615-8a84-4c75863056fa admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:21.903454 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785b528-ee1b-4615-8a84-4c75863056fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:21.903798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785b528-ee1b-4615-8a84-4c75863056fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:21.904536 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785b528-ee1b-4615-8a84-4c75863056fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:21.907180 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785b528-ee1b-4615-8a84-4c75863056fa admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:21.915663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce72176-996d-4823-bc62-773e093a65d2 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:21.919140 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-0ce72176-996d-4823-bc62-773e093a65d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:21.957970 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce72176-996d-4823-bc62-773e093a65d2 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:21.964413 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f54dfd-0974-446f-b34f-28cd4aec6c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:21.965293 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f54dfd-0974-446f-b34f-28cd4aec6c9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:21.965769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f54dfd-0974-446f-b34f-28cd4aec6c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:21.966140 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f54dfd-0974-446f-b34f-28cd4aec6c9f admin admin] PATH_INFO: `/v3/projects/bc8e3c578de04aab9bc406861ffab4a7` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:21.967483 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f54dfd-0974-446f-b34f-28cd4aec6c9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc8e3c578de04aab9bc406861ffab4a7)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:21.975558 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f54dfd-0974-446f-b34f-28cd4aec6c9f admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:21.995495 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:22.030401 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1065/2140] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:14:21 2022] POST /identity/v3/projects => generated 383 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:22.035037 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785b528-ee1b-4615-8a84-4c75863056fa admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:22.103026 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785b528-ee1b-4615-8a84-4c75863056fa admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:22.108101 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d48047a-805e-4daf-a656-da1f175ba662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:22.108796 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48047a-805e-4daf-a656-da1f175ba662 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:22.109216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48047a-805e-4daf-a656-da1f175ba662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:22.109641 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48047a-805e-4daf-a656-da1f175ba662 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:22.110513 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d48047a-805e-4daf-a656-da1f175ba662 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:22.112425 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d48047a-805e-4daf-a656-da1f175ba662 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:22.139145 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-45f54dfd-0974-446f-b34f-28cd4aec6c9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc8e3c578de04aab9bc406861ffab4a7', 'request_id': 'req-45f54dfd-0974-446f-b34f-28cd4aec6c9f'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:22.149719 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-45f54dfd-0974-446f-b34f-28cd4aec6c9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc8e3c578de04aab9bc406861ffab4a7', 'request_id': 'req-45f54dfd-0974-446f-b34f-28cd4aec6c9f'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:22.217572 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1066/2141] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:14:22 2022] POST /identity/v3/users => generated 292 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:22.218916 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d48047a-805e-4daf-a656-da1f175ba662 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:22.229235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d48047a-805e-4daf-a656-da1f175ba662 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:22.237063 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-45f54dfd-0974-446f-b34f-28cd4aec6c9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc8e3c578de04aab9bc406861ffab4a7', 'request_id': 'req-45f54dfd-0974-446f-b34f-28cd4aec6c9f'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:22.249883 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-45f54dfd-0974-446f-b34f-28cd4aec6c9f admin admin] The token cache is being invalidate because project bc8e3c578de04aab9bc406861ffab4a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:14:22.249883 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-45f54dfd-0974-446f-b34f-28cd4aec6c9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:22.251539 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-45f54dfd-0974-446f-b34f-28cd4aec6c9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:22.253685 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1076/2142] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:14:21 2022] DELETE /identity/v3/projects/bc8e3c578de04aab9bc406861ffab4a7 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:22.257521 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f54dfd-0974-446f-b34f-28cd4aec6c9f admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:22.259733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-45f54dfd-0974-446f-b34f-28cd4aec6c9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:22.295236 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e85f78-ad00-459c-9fda-79c83dcf6a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:22.295888 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e85f78-ad00-459c-9fda-79c83dcf6a84 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:22.296170 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e85f78-ad00-459c-9fda-79c83dcf6a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:22.296444 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e85f78-ad00-459c-9fda-79c83dcf6a84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:22.297317 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e85f78-ad00-459c-9fda-79c83dcf6a84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:22.299072 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e85f78-ad00-459c-9fda-79c83dcf6a84 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:22.309076 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1067/2143] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:14:22.313649 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e85f78-ad00-459c-9fda-79c83dcf6a84 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:22.315476 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-59e85f78-ad00-459c-9fda-79c83dcf6a84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:22.327481 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e85f78-ad00-459c-9fda-79c83dcf6a84 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:22.330888 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f54dfd-0974-446f-b34f-28cd4aec6c9f admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:22.333575 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe973524-1a79-4846-804e-7118d2826480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:22.334581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe973524-1a79-4846-804e-7118d2826480 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:22.334690 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe973524-1a79-4846-804e-7118d2826480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:22.334929 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe973524-1a79-4846-804e-7118d2826480 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:22.335891 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe973524-1a79-4846-804e-7118d2826480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:22.337452 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe973524-1a79-4846-804e-7118d2826480 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:22.338643 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bde557-9bac-4cfb-9407-4462b868cb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:22.339493 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bde557-9bac-4cfb-9407-4462b868cb2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:22.339783 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bde557-9bac-4cfb-9407-4462b868cb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:22.340150 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bde557-9bac-4cfb-9407-4462b868cb2b admin admin] PATH_INFO: `/v3/users/228c56bccacb4ddca847f799fc24dd12` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:22.341360 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bde557-9bac-4cfb-9407-4462b868cb2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=228c56bccacb4ddca847f799fc24dd12)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:22.343420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bde557-9bac-4cfb-9407-4462b868cb2b admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:22.355068 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1068/2144] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:14:22.357621 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe973524-1a79-4846-804e-7118d2826480 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:22.374249 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe973524-1a79-4846-804e-7118d2826480 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:22.379039 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e529ebd-6352-48f2-bc49-48b6c826d166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:22.379764 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e529ebd-6352-48f2-bc49-48b6c826d166 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:22.380044 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e529ebd-6352-48f2-bc49-48b6c826d166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:22.380469 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e529ebd-6352-48f2-bc49-48b6c826d166 admin admin] PATH_INFO: `/v3/projects/5121fbd6cff74504b78c2bfdd4fedf85/users/0f97649f8a924b68b3c46e1723c328eb/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:22.401991 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e529ebd-6352-48f2-bc49-48b6c826d166 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5121fbd6cff74504b78c2bfdd4fedf85, user_id=0f97649f8a924b68b3c46e1723c328eb, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:22.405207 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e529ebd-6352-48f2-bc49-48b6c826d166 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:22.476632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1069/2145] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:14:22 2022] PUT /identity/v3/projects/5121fbd6cff74504b78c2bfdd4fedf85/users/0f97649f8a924b68b3c46e1723c328eb/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:22.487897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe52fc-4692-4b0c-b6b4-d89dc7fafd04 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:22.488253 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe52fc-4692-4b0c-b6b4-d89dc7fafd04 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:22.488594 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe52fc-4692-4b0c-b6b4-d89dc7fafd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:22.520237 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-44fe52fc-4692-4b0c-b6b4-d89dc7fafd04 None None] MFA Rules not processed for user `0f97649f8a924b68b3c46e1723c328eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:22.521440 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-44fe52fc-4692-4b0c-b6b4-d89dc7fafd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:22.528932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-62bde557-9bac-4cfb-9407-4462b868cb2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '228c56bccacb4ddca847f799fc24dd12'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:22.586021 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1070/2146] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:14:22.590545 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe52fc-4692-4b0c-b6b4-d89dc7fafd04 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:22.631147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-62bde557-9bac-4cfb-9407-4462b868cb2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '228c56bccacb4ddca847f799fc24dd12'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:22.631835 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-62bde557-9bac-4cfb-9407-4462b868cb2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '228c56bccacb4ddca847f799fc24dd12'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:22.657757 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-62bde557-9bac-4cfb-9407-4462b868cb2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '228c56bccacb4ddca847f799fc24dd12'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:22.658661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-62bde557-9bac-4cfb-9407-4462b868cb2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '228c56bccacb4ddca847f799fc24dd12'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:22.686138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe52fc-4692-4b0c-b6b4-d89dc7fafd04 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:22.691611 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1077/2147] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:14:22 2022] DELETE /identity/v3/users/228c56bccacb4ddca847f799fc24dd12 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:22.694359 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e5aec-73df-46b5-97d3-35e78d39e1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:22.694359 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e5aec-73df-46b5-97d3-35e78d39e1fd admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:22.694359 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e5aec-73df-46b5-97d3-35e78d39e1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:22.694359 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e5aec-73df-46b5-97d3-35e78d39e1fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:22.694875 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916e5aec-73df-46b5-97d3-35e78d39e1fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:22.696197 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916e5aec-73df-46b5-97d3-35e78d39e1fd admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:22.865558 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1071/2148] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:14:22 2022] POST /identity/v3/projects => generated 383 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:14:22.869726 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bde557-9bac-4cfb-9407-4462b868cb2b admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:22.871229 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-62bde557-9bac-4cfb-9407-4462b868cb2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:22.954376 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bde557-9bac-4cfb-9407-4462b868cb2b admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:23.031790 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888472ad-19b9-424a-9972-45fb9f0c3cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:23.033891 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888472ad-19b9-424a-9972-45fb9f0c3cc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:23.034471 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888472ad-19b9-424a-9972-45fb9f0c3cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:23.034923 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888472ad-19b9-424a-9972-45fb9f0c3cc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:23.035753 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888472ad-19b9-424a-9972-45fb9f0c3cc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:23.038058 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888472ad-19b9-424a-9972-45fb9f0c3cc8 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:23.174673 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1078/2149] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:14:22 2022] POST /identity/v3/users => generated 292 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:23.178689 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e5aec-73df-46b5-97d3-35e78d39e1fd admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:23.194632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e5aec-73df-46b5-97d3-35e78d39e1fd admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:23.197096 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b2d41-4d6d-467d-87f1-79c772dc12cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:23.197747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b2d41-4d6d-467d-87f1-79c772dc12cc admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:23.198028 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b2d41-4d6d-467d-87f1-79c772dc12cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:23.198300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b2d41-4d6d-467d-87f1-79c772dc12cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:23.199059 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b2d41-4d6d-467d-87f1-79c772dc12cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:23.200659 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b2d41-4d6d-467d-87f1-79c772dc12cc admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:23.211160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1072/2150] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:14:23.215252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888472ad-19b9-424a-9972-45fb9f0c3cc8 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:23.226986 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888472ad-19b9-424a-9972-45fb9f0c3cc8 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:23.227694 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b2d41-4d6d-467d-87f1-79c772dc12cc admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:23.229754 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-5c5b2d41-4d6d-467d-87f1-79c772dc12cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:23.233318 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450fc42-4957-48d1-8904-c14c8191a2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:23.233958 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450fc42-4957-48d1-8904-c14c8191a2a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:23.235087 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450fc42-4957-48d1-8904-c14c8191a2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:23.237644 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450fc42-4957-48d1-8904-c14c8191a2a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:23.238670 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450fc42-4957-48d1-8904-c14c8191a2a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:23.242194 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450fc42-4957-48d1-8904-c14c8191a2a0 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:23.247237 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b2d41-4d6d-467d-87f1-79c772dc12cc admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:23.259267 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1079/2151] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:14:23.259969 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4eb83e-403d-4bb8-a488-d8c30e56574d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:23.260174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4eb83e-403d-4bb8-a488-d8c30e56574d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:23.260738 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4eb83e-403d-4bb8-a488-d8c30e56574d admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:23.260854 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4eb83e-403d-4bb8-a488-d8c30e56574d admin admin] PATH_INFO: `/v3/projects/b5e409d5fe204d2cb86fab77ca60ab41` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:23.262080 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4eb83e-403d-4bb8-a488-d8c30e56574d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5e409d5fe204d2cb86fab77ca60ab41)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:23.262404 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450fc42-4957-48d1-8904-c14c8191a2a0 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:23.264463 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4eb83e-403d-4bb8-a488-d8c30e56574d admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:23.274461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450fc42-4957-48d1-8904-c14c8191a2a0 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:23.279676 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be61505-9d1c-47bf-96d1-f836052579af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:23.280371 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be61505-9d1c-47bf-96d1-f836052579af admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:23.280664 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be61505-9d1c-47bf-96d1-f836052579af admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:23.280939 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be61505-9d1c-47bf-96d1-f836052579af admin admin] PATH_INFO: `/v3/projects/2e85a0ae928f421fb2fec1285c4c1941/users/3ae4f616190147ef9ed4e19b6317006d/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:23.343990 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be61505-9d1c-47bf-96d1-f836052579af admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e85a0ae928f421fb2fec1285c4c1941, user_id=3ae4f616190147ef9ed4e19b6317006d, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:23.347194 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be61505-9d1c-47bf-96d1-f836052579af admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:23.470259 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1080/2152] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:14:23 2022] PUT /identity/v3/projects/2e85a0ae928f421fb2fec1285c4c1941/users/3ae4f616190147ef9ed4e19b6317006d/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:23.482952 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6a193-31b0-4f93-bc02-e80dd98634ab None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:23.483165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6a193-31b0-4f93-bc02-e80dd98634ab None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:23.483495 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6a193-31b0-4f93-bc02-e80dd98634ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:23.483672 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8a4eb83e-403d-4bb8-a488-d8c30e56574d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5e409d5fe204d2cb86fab77ca60ab41', 'request_id': 'req-8a4eb83e-403d-4bb8-a488-d8c30e56574d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:23.494026 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:23.514554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-c1b6a193-31b0-4f93-bc02-e80dd98634ab None None] MFA Rules not processed for user `3ae4f616190147ef9ed4e19b6317006d`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:23.516415 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-c1b6a193-31b0-4f93-bc02-e80dd98634ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:23.580331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8a4eb83e-403d-4bb8-a488-d8c30e56574d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5e409d5fe204d2cb86fab77ca60ab41', 'request_id': 'req-8a4eb83e-403d-4bb8-a488-d8c30e56574d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:23.581045 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:23.592084 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8a4eb83e-403d-4bb8-a488-d8c30e56574d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5e409d5fe204d2cb86fab77ca60ab41', 'request_id': 'req-8a4eb83e-403d-4bb8-a488-d8c30e56574d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:23.596397 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1081/2153] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:14:23.601531 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b6a193-31b0-4f93-bc02-e80dd98634ab None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:23.603504 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8a4eb83e-403d-4bb8-a488-d8c30e56574d admin admin] The token cache is being invalidate because project b5e409d5fe204d2cb86fab77ca60ab41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:14:23.603808 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8a4eb83e-403d-4bb8-a488-d8c30e56574d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:23.604470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8a4eb83e-403d-4bb8-a488-d8c30e56574d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:23.606390 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1073/2154] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:14:23 2022] DELETE /identity/v3/projects/b5e409d5fe204d2cb86fab77ca60ab41 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:23.610998 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6bc146-3aec-42dd-b08e-3c846f631ed1 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:23.611316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6bc146-3aec-42dd-b08e-3c846f631ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:23.611596 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6bc146-3aec-42dd-b08e-3c846f631ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:23.633283 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-ca6bc146-3aec-42dd-b08e-3c846f631ed1 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:23.635569 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ca6bc146-3aec-42dd-b08e-3c846f631ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:23.672344 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b6a193-31b0-4f93-bc02-e80dd98634ab None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:23.677173 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd4661a-133f-4b22-8a38-3b28c8c77d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:23.677791 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4661a-133f-4b22-8a38-3b28c8c77d95 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:23.678076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4661a-133f-4b22-8a38-3b28c8c77d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:23.678349 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4661a-133f-4b22-8a38-3b28c8c77d95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:23.679044 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd4661a-133f-4b22-8a38-3b28c8c77d95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:23.681176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd4661a-133f-4b22-8a38-3b28c8c77d95 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:23.705339 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1074/2155] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:14:23.712808 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406f51b-6bd3-45d5-a0fb-fff655a22fb4 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:23.713101 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406f51b-6bd3-45d5-a0fb-fff655a22fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:23.713386 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406f51b-6bd3-45d5-a0fb-fff655a22fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:23.732793 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-5406f51b-6bd3-45d5-a0fb-fff655a22fb4 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:23.733995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-5406f51b-6bd3-45d5-a0fb-fff655a22fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:23.741980 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1075/2156] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:14:23.745953 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406f51b-6bd3-45d5-a0fb-fff655a22fb4 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:23.755335 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406f51b-6bd3-45d5-a0fb-fff655a22fb4 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:23.760183 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a355d5-aff1-4d1c-b471-456bc5b17481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:23.760782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a355d5-aff1-4d1c-b471-456bc5b17481 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:23.761065 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a355d5-aff1-4d1c-b471-456bc5b17481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:23.765196 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a355d5-aff1-4d1c-b471-456bc5b17481 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:23.765196 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a355d5-aff1-4d1c-b471-456bc5b17481 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:14:23.765196 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a355d5-aff1-4d1c-b471-456bc5b17481 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:23.765196 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a355d5-aff1-4d1c-b471-456bc5b17481 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:23.771426 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1082/2157] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:14:23 2022] POST /identity/v3/projects => generated 383 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:23.772969 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd4661a-133f-4b22-8a38-3b28c8c77d95 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:23.774884 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-3dd4661a-133f-4b22-8a38-3b28c8c77d95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:23.776562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1076/2158] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 05:14:23.780687 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a355d5-aff1-4d1c-b471-456bc5b17481 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:23.791497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a355d5-aff1-4d1c-b471-456bc5b17481 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:23.839157 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd4661a-133f-4b22-8a38-3b28c8c77d95 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:23.843866 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2a436-e128-492c-8c4c-e2a5ce493772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:23.844480 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2a436-e128-492c-8c4c-e2a5ce493772 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:23.844761 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2a436-e128-492c-8c4c-e2a5ce493772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:23.845037 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2a436-e128-492c-8c4c-e2a5ce493772 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:23.845699 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2a436-e128-492c-8c4c-e2a5ce493772 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:23.847984 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2a436-e128-492c-8c4c-e2a5ce493772 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:23.849470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac484b86-2388-41d9-92d9-53c17b8f36ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:23.850695 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac484b86-2388-41d9-92d9-53c17b8f36ab admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:23.850695 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac484b86-2388-41d9-92d9-53c17b8f36ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:23.850695 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac484b86-2388-41d9-92d9-53c17b8f36ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:23.851454 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac484b86-2388-41d9-92d9-53c17b8f36ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:23.853352 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac484b86-2388-41d9-92d9-53c17b8f36ab admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:23.955824 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1083/2159] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:14:23 2022] POST /identity/v3/users => generated 298 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:14:23.959615 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2a436-e128-492c-8c4c-e2a5ce493772 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:23.969925 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2a436-e128-492c-8c4c-e2a5ce493772 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:23.974091 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1077/2160] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:14:23 2022] POST /identity/v3/projects => generated 389 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:14:23.975214 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac484b86-2388-41d9-92d9-53c17b8f36ab admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:24.032741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f44a59e-5f7f-4551-ad79-bf642f88c0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:24.034711 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44a59e-5f7f-4551-ad79-bf642f88c0f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:24.035037 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44a59e-5f7f-4551-ad79-bf642f88c0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:24.035318 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44a59e-5f7f-4551-ad79-bf642f88c0f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:24.036024 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f44a59e-5f7f-4551-ad79-bf642f88c0f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:24.037656 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f44a59e-5f7f-4551-ad79-bf642f88c0f2 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:24.042829 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac484b86-2388-41d9-92d9-53c17b8f36ab admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:24.047779 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60668a29-0bd3-4591-948f-42e6360c3479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:24.048427 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60668a29-0bd3-4591-948f-42e6360c3479 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:24.048764 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60668a29-0bd3-4591-948f-42e6360c3479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:24.049051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60668a29-0bd3-4591-948f-42e6360c3479 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:24.052887 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1084/2161] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:14:24.053018 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60668a29-0bd3-4591-948f-42e6360c3479 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:24.053096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f44a59e-5f7f-4551-ad79-bf642f88c0f2 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:24.053192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60668a29-0bd3-4591-948f-42e6360c3479 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:24.054218 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-4f44a59e-5f7f-4551-ad79-bf642f88c0f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:24.068772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f44a59e-5f7f-4551-ad79-bf642f88c0f2 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:24.074936 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85845b2c-951e-4ae5-a6b9-0ce4ca9dbcc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:24.074936 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85845b2c-951e-4ae5-a6b9-0ce4ca9dbcc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:24.074936 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85845b2c-951e-4ae5-a6b9-0ce4ca9dbcc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:24.074936 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85845b2c-951e-4ae5-a6b9-0ce4ca9dbcc2 admin admin] PATH_INFO: `/v3/users/3cc20dc10ec74ae1bd814b183991e3a2` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:24.076006 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85845b2c-951e-4ae5-a6b9-0ce4ca9dbcc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cc20dc10ec74ae1bd814b183991e3a2)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:24.078543 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85845b2c-951e-4ae5-a6b9-0ce4ca9dbcc2 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:24.128571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1078/2162] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:14:23 2022] POST /identity/v3/users => generated 295 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:14:24.132488 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60668a29-0bd3-4591-948f-42e6360c3479 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:24.142975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60668a29-0bd3-4591-948f-42e6360c3479 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:24.148111 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de4d53-d664-4e18-be54-4b536bf894e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:24.148781 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de4d53-d664-4e18-be54-4b536bf894e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:24.149065 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de4d53-d664-4e18-be54-4b536bf894e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:24.160779 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de4d53-d664-4e18-be54-4b536bf894e0 admin admin] PATH_INFO: `/v3/projects/e20e5ce9e0914a64a918dbcd9e541b11/users/035cf1925dd84867a0e68fc392023b17/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:24.166030 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90de4d53-d664-4e18-be54-4b536bf894e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e20e5ce9e0914a64a918dbcd9e541b11, user_id=035cf1925dd84867a0e68fc392023b17, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:24.168895 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90de4d53-d664-4e18-be54-4b536bf894e0 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:24.225292 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:24.253927 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1079/2163] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:14:24 2022] PUT /identity/v3/projects/e20e5ce9e0914a64a918dbcd9e541b11/users/035cf1925dd84867a0e68fc392023b17/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:24.254984 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de4d53-d664-4e18-be54-4b536bf894e0 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:24.319850 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-85845b2c-951e-4ae5-a6b9-0ce4ca9dbcc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cc20dc10ec74ae1bd814b183991e3a2'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:24.320750 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de4d53-d664-4e18-be54-4b536bf894e0 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:24.326241 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b61c18-f365-4cee-93a8-b965eb3af1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:24.326900 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b61c18-f365-4cee-93a8-b965eb3af1d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:24.327235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b61c18-f365-4cee-93a8-b965eb3af1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:24.327504 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b61c18-f365-4cee-93a8-b965eb3af1d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:24.328762 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b61c18-f365-4cee-93a8-b965eb3af1d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:24.330366 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b61c18-f365-4cee-93a8-b965eb3af1d0 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:24.340951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1080/2164] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:14:24.342041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b61c18-f365-4cee-93a8-b965eb3af1d0 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:24.353334 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b61c18-f365-4cee-93a8-b965eb3af1d0 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:24.358313 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879b216-ecc7-4bd4-b7d7-6c501213ab72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:24.358955 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879b216-ecc7-4bd4-b7d7-6c501213ab72 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:24.363848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879b216-ecc7-4bd4-b7d7-6c501213ab72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:24.363848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879b216-ecc7-4bd4-b7d7-6c501213ab72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:24.363848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9879b216-ecc7-4bd4-b7d7-6c501213ab72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:24.363848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9879b216-ecc7-4bd4-b7d7-6c501213ab72 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:24.373017 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1081/2165] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:14:24.374046 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879b216-ecc7-4bd4-b7d7-6c501213ab72 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:24.393941 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879b216-ecc7-4bd4-b7d7-6c501213ab72 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:24.398778 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5a3db-0ac4-4556-9fd4-cc31c85ef419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:24.403364 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5a3db-0ac4-4556-9fd4-cc31c85ef419 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:24.403364 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5a3db-0ac4-4556-9fd4-cc31c85ef419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:24.403364 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5a3db-0ac4-4556-9fd4-cc31c85ef419 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:24.403364 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f5a3db-0ac4-4556-9fd4-cc31c85ef419 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:24.403364 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f5a3db-0ac4-4556-9fd4-cc31c85ef419 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:24.413815 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1082/2166] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:14:24.414586 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5a3db-0ac4-4556-9fd4-cc31c85ef419 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:24.422654 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-85845b2c-951e-4ae5-a6b9-0ce4ca9dbcc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cc20dc10ec74ae1bd814b183991e3a2'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:24.423327 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-85845b2c-951e-4ae5-a6b9-0ce4ca9dbcc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cc20dc10ec74ae1bd814b183991e3a2'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:24.424048 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5a3db-0ac4-4556-9fd4-cc31c85ef419 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:24.429118 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f687a86-e6fd-40cb-b5d8-410cce8633b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:24.429777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f687a86-e6fd-40cb-b5d8-410cce8633b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:24.430165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f687a86-e6fd-40cb-b5d8-410cce8633b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:24.430339 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f687a86-e6fd-40cb-b5d8-410cce8633b1 admin admin] PATH_INFO: `/v3/projects/e20e5ce9e0914a64a918dbcd9e541b11/users/035cf1925dd84867a0e68fc392023b17/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:24.432379 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f687a86-e6fd-40cb-b5d8-410cce8633b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e20e5ce9e0914a64a918dbcd9e541b11, user_id=035cf1925dd84867a0e68fc392023b17, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:24.435137 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f687a86-e6fd-40cb-b5d8-410cce8633b1 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:24.436580 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-85845b2c-951e-4ae5-a6b9-0ce4ca9dbcc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cc20dc10ec74ae1bd814b183991e3a2'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:24.437252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-85845b2c-951e-4ae5-a6b9-0ce4ca9dbcc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cc20dc10ec74ae1bd814b183991e3a2'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:24.445977 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1083/2167] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:14:24 2022] PUT /identity/v3/projects/e20e5ce9e0914a64a918dbcd9e541b11/users/035cf1925dd84867a0e68fc392023b17/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:24.446762 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f687a86-e6fd-40cb-b5d8-410cce8633b1 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:24.448846 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6f687a86-e6fd-40cb-b5d8-410cce8633b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:24.451145 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1085/2168] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:14:24 2022] DELETE /identity/v3/users/3cc20dc10ec74ae1bd814b183991e3a2 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:24.455949 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97692b-119d-4a10-9e92-a5402785544e None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:24.456322 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97692b-119d-4a10-9e92-a5402785544e None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:24.456666 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97692b-119d-4a10-9e92-a5402785544e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:24.477491 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-7c97692b-119d-4a10-9e92-a5402785544e None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:24.479102 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-7c97692b-119d-4a10-9e92-a5402785544e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:24.525717 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f687a86-e6fd-40cb-b5d8-410cce8633b1 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:24.546656 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1086/2169] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:14:24.548854 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c79e3-332c-4aa1-b4d2-34f9eb662748 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:24.549221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c79e3-332c-4aa1-b4d2-34f9eb662748 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:24.549580 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c79e3-332c-4aa1-b4d2-34f9eb662748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:24.580369 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-c93c79e3-332c-4aa1-b4d2-34f9eb662748 None None] MFA Rules not processed for user `035cf1925dd84867a0e68fc392023b17`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:24.581697 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-c93c79e3-332c-4aa1-b4d2-34f9eb662748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:24.593287 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a3ed8-a419-4af6-a01a-a9531d20aeff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:24.593287 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a3ed8-a419-4af6-a01a-a9531d20aeff admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:24.593287 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a3ed8-a419-4af6-a01a-a9531d20aeff admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:24.593287 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a3ed8-a419-4af6-a01a-a9531d20aeff admin admin] PATH_INFO: `/v3/projects/b3fa064fa97f4831b167b339a52ea987/users/f0606defb69a4badb994f764e0916802/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:24.593662 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:24.609774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7a3ed8-a419-4af6-a01a-a9531d20aeff admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3fa064fa97f4831b167b339a52ea987, user_id=f0606defb69a4badb994f764e0916802, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:24.613367 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7a3ed8-a419-4af6-a01a-a9531d20aeff admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:24.687034 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1087/2170] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:24 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:24.688454 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c79e3-332c-4aa1-b4d2-34f9eb662748 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:24.689955 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-c93c79e3-332c-4aa1-b4d2-34f9eb662748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:24.730573 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:24.736236 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1084/2171] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:14:24 2022] PUT /identity/v3/projects/b3fa064fa97f4831b167b339a52ea987/users/f0606defb69a4badb994f764e0916802/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:24.744865 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a3ed8-a419-4af6-a01a-a9531d20aeff admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:24.744865 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7e7a3ed8-a419-4af6-a01a-a9531d20aeff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:24.762526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c79e3-332c-4aa1-b4d2-34f9eb662748 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:24.791203 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:24.801865 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a3ed8-a419-4af6-a01a-a9531d20aeff admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:24.806261 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea4fd7-c7a5-478a-904a-be1f11ca62bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:24.807012 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea4fd7-c7a5-478a-904a-be1f11ca62bb service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:24.807502 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea4fd7-c7a5-478a-904a-be1f11ca62bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:24.807911 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea4fd7-c7a5-478a-904a-be1f11ca62bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:24.809897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-81ea4fd7-c7a5-478a-904a-be1f11ca62bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:24.818191 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ba89e-d668-495d-a51d-d7bd5f998b74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:24.818827 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ba89e-d668-495d-a51d-d7bd5f998b74 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:24.819147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ba89e-d668-495d-a51d-d7bd5f998b74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:24.819429 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ba89e-d668-495d-a51d-d7bd5f998b74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:24.872969 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ea4fd7-c7a5-478a-904a-be1f11ca62bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:24.874423 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ea4fd7-c7a5-478a-904a-be1f11ca62bb service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:24.882408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ba89e-d668-495d-a51d-d7bd5f998b74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:24.883928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ba89e-d668-495d-a51d-d7bd5f998b74 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:24.884063 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1085/2172] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:14:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:14:24.885309 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f85cf-c7b1-4692-9693-035e45e249fe None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:24.885636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f85cf-c7b1-4692-9693-035e45e249fe None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:24.885923 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f85cf-c7b1-4692-9693-035e45e249fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:24.894888 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1088/2173] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:14:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:14:24.917852 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:24.920652 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-002f85cf-c7b1-4692-9693-035e45e249fe None None] MFA Rules not processed for user `f0606defb69a4badb994f764e0916802`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:24.922670 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-002f85cf-c7b1-4692-9693-035e45e249fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:25.001835 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1086/2174] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:24 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:25.006490 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ba89e-d668-495d-a51d-d7bd5f998b74 service neutron] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:25.016790 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ba89e-d668-495d-a51d-d7bd5f998b74 service neutron] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:25.022336 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60029e88-c60c-4a5e-bd05-a9a1abdc083b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:25.023182 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60029e88-c60c-4a5e-bd05-a9a1abdc083b admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:25.023471 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60029e88-c60c-4a5e-bd05-a9a1abdc083b admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:25.023748 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60029e88-c60c-4a5e-bd05-a9a1abdc083b admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:25.024417 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60029e88-c60c-4a5e-bd05-a9a1abdc083b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:25.026653 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60029e88-c60c-4a5e-bd05-a9a1abdc083b admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:25.153107 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:25.164184 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1089/2175] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:14:25 2022] POST /identity/v3/projects => generated 387 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:25.165468 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f85cf-c7b1-4692-9693-035e45e249fe None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:25.179693 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f85cf-c7b1-4692-9693-035e45e249fe None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:25.246696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5337c4b-272d-405f-84d8-8dc7fd7d2b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:25.247368 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5337c4b-272d-405f-84d8-8dc7fd7d2b92 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:25.247658 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5337c4b-272d-405f-84d8-8dc7fd7d2b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:25.247940 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5337c4b-272d-405f-84d8-8dc7fd7d2b92 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:25.248619 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5337c4b-272d-405f-84d8-8dc7fd7d2b92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:25.254036 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5337c4b-272d-405f-84d8-8dc7fd7d2b92 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:25.312920 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60029e88-c60c-4a5e-bd05-a9a1abdc083b admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:25.314715 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-60029e88-c60c-4a5e-bd05-a9a1abdc083b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:25.327740 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60029e88-c60c-4a5e-bd05-a9a1abdc083b admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:25.333281 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1decc508-791a-42e1-be18-e3c66e02f1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:25.333930 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decc508-791a-42e1-be18-e3c66e02f1ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:25.334213 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decc508-791a-42e1-be18-e3c66e02f1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:25.334488 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decc508-791a-42e1-be18-e3c66e02f1ad admin admin] PATH_INFO: `/v3/projects/5768143089fe4d1d9cffd180c16b88a1` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:25.335727 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1decc508-791a-42e1-be18-e3c66e02f1ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=5768143089fe4d1d9cffd180c16b88a1)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:25.337657 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1decc508-791a-42e1-be18-e3c66e02f1ad admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:25.375752 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:25.384827 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1087/2176] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:14:25 2022] POST /identity/v3/users => generated 300 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:25.385992 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5337c4b-272d-405f-84d8-8dc7fd7d2b92 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:25.397450 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5337c4b-272d-405f-84d8-8dc7fd7d2b92 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:25.402908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e477d-1668-4119-8953-692c839a139e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:25.403530 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e477d-1668-4119-8953-692c839a139e admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:25.403814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e477d-1668-4119-8953-692c839a139e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:25.404091 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e477d-1668-4119-8953-692c839a139e admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:25.404824 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46e477d-1668-4119-8953-692c839a139e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:25.406490 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46e477d-1668-4119-8953-692c839a139e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:25.417060 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1088/2177] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:14:25.421446 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e477d-1668-4119-8953-692c839a139e admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:25.431607 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e477d-1668-4119-8953-692c839a139e admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:25.432884 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-1decc508-791a-42e1-be18-e3c66e02f1ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5768143089fe4d1d9cffd180c16b88a1', 'request_id': 'req-1decc508-791a-42e1-be18-e3c66e02f1ad'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:25.442975 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-1decc508-791a-42e1-be18-e3c66e02f1ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5768143089fe4d1d9cffd180c16b88a1', 'request_id': 'req-1decc508-791a-42e1-be18-e3c66e02f1ad'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:25.492102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad357e60-4827-46a5-8c77-2d6b83b06bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:25.493921 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-1decc508-791a-42e1-be18-e3c66e02f1ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5768143089fe4d1d9cffd180c16b88a1', 'request_id': 'req-1decc508-791a-42e1-be18-e3c66e02f1ad'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:25.495126 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad357e60-4827-46a5-8c77-2d6b83b06bce admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:25.495805 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad357e60-4827-46a5-8c77-2d6b83b06bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:25.496256 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad357e60-4827-46a5-8c77-2d6b83b06bce admin admin] PATH_INFO: `/v3/projects/cd947cc5313444e2b22b0c56cc55e812/users/331b2ee0c8f74548a206d50cadd4f813/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:25.507322 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-1decc508-791a-42e1-be18-e3c66e02f1ad admin admin] The token cache is being invalidate because project 5768143089fe4d1d9cffd180c16b88a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:14:25.507566 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-1decc508-791a-42e1-be18-e3c66e02f1ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:25.508231 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-1decc508-791a-42e1-be18-e3c66e02f1ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:25.510015 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1090/2178] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:14:25 2022] DELETE /identity/v3/projects/5768143089fe4d1d9cffd180c16b88a1 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:14:25.514714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad357e60-4827-46a5-8c77-2d6b83b06bce admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd947cc5313444e2b22b0c56cc55e812, user_id=331b2ee0c8f74548a206d50cadd4f813, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:25.517702 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff849e-61fb-4832-affc-1aeb1d95d92e None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:25.517702 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff849e-61fb-4832-affc-1aeb1d95d92e None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:25.517702 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff849e-61fb-4832-affc-1aeb1d95d92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:25.518088 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad357e60-4827-46a5-8c77-2d6b83b06bce admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:25.535612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-d8ff849e-61fb-4832-affc-1aeb1d95d92e None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:25.536844 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-d8ff849e-61fb-4832-affc-1aeb1d95d92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:25.544820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1091/2179] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:14:25.553409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c3e6c-2fa9-4d8b-b780-ef9216292c6e None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:25.553700 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c3e6c-2fa9-4d8b-b780-ef9216292c6e None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:25.553981 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c3e6c-2fa9-4d8b-b780-ef9216292c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:25.573766 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-392c3e6c-2fa9-4d8b-b780-ef9216292c6e None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:25.575109 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-392c3e6c-2fa9-4d8b-b780-ef9216292c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:25.582892 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1092/2180] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:14:25.586955 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c3e6c-2fa9-4d8b-b780-ef9216292c6e None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:25.596263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c3e6c-2fa9-4d8b-b780-ef9216292c6e None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:25.602990 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1089/2181] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:14:25 2022] PUT /identity/v3/projects/cd947cc5313444e2b22b0c56cc55e812/users/331b2ee0c8f74548a206d50cadd4f813/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:14:25.606271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad357e60-4827-46a5-8c77-2d6b83b06bce admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:25.607991 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ad357e60-4827-46a5-8c77-2d6b83b06bce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:25.656134 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5597198f-c833-40e7-874c-aad01df65925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:25.657193 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597198f-c833-40e7-874c-aad01df65925 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:25.657482 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597198f-c833-40e7-874c-aad01df65925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:25.657766 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597198f-c833-40e7-874c-aad01df65925 admin admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:25.658466 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5597198f-c833-40e7-874c-aad01df65925 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:14:25.658780 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5597198f-c833-40e7-874c-aad01df65925 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:25.662289 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5597198f-c833-40e7-874c-aad01df65925 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:25.679374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1093/2182] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 05:14:25.691942 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5597198f-c833-40e7-874c-aad01df65925 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:25.692426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad357e60-4827-46a5-8c77-2d6b83b06bce admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:25.692426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e426420-1113-4e9e-8de8-9ebfd840c621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:25.692426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e426420-1113-4e9e-8de8-9ebfd840c621 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:25.692664 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e426420-1113-4e9e-8de8-9ebfd840c621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:25.692953 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e426420-1113-4e9e-8de8-9ebfd840c621 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:25.693914 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e426420-1113-4e9e-8de8-9ebfd840c621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:25.695646 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e426420-1113-4e9e-8de8-9ebfd840c621 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:25.696183 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5597198f-c833-40e7-874c-aad01df65925 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:25.702710 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6565ff3-d464-4d4a-b442-8587fe6c0db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:25.703934 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6565ff3-d464-4d4a-b442-8587fe6c0db0 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:25.704957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6565ff3-d464-4d4a-b442-8587fe6c0db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:25.705368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6565ff3-d464-4d4a-b442-8587fe6c0db0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:25.706191 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6565ff3-d464-4d4a-b442-8587fe6c0db0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:25.707623 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1090/2183] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:14:25.708902 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6565ff3-d464-4d4a-b442-8587fe6c0db0 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:25.723934 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e426420-1113-4e9e-8de8-9ebfd840c621 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:25.738076 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e426420-1113-4e9e-8de8-9ebfd840c621 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:25.743428 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c3459-0307-43ee-9582-7d7618716016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:25.744366 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c3459-0307-43ee-9582-7d7618716016 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:25.745083 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c3459-0307-43ee-9582-7d7618716016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:25.747448 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c3459-0307-43ee-9582-7d7618716016 admin admin] PATH_INFO: `/v3/projects/cd947cc5313444e2b22b0c56cc55e812/users/331b2ee0c8f74548a206d50cadd4f813/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:25.749005 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4c3459-0307-43ee-9582-7d7618716016 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd947cc5313444e2b22b0c56cc55e812, user_id=331b2ee0c8f74548a206d50cadd4f813, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:25.752286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4c3459-0307-43ee-9582-7d7618716016 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:25.765205 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1091/2184] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:14:25 2022] PUT /identity/v3/projects/cd947cc5313444e2b22b0c56cc55e812/users/331b2ee0c8f74548a206d50cadd4f813/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:25.776852 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92ac08-523c-4482-b78c-3311fe14e4df None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:25.777150 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92ac08-523c-4482-b78c-3311fe14e4df None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:25.777557 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92ac08-523c-4482-b78c-3311fe14e4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:25.815989 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-8c92ac08-523c-4482-b78c-3311fe14e4df None None] MFA Rules not processed for user `331b2ee0c8f74548a206d50cadd4f813`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:25.816720 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8c92ac08-523c-4482-b78c-3311fe14e4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:25.826480 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1094/2185] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:14:25 2022] POST /identity/v3/projects => generated 375 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:25.831450 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6565ff3-d464-4d4a-b442-8587fe6c0db0 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:25.933717 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1092/2186] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:25 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:14:25.939981 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92ac08-523c-4482-b78c-3311fe14e4df None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:25.941788 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8c92ac08-523c-4482-b78c-3311fe14e4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:25.963182 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6565ff3-d464-4d4a-b442-8587fe6c0db0 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:25.968279 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cde88f3-8006-4c6c-96dd-50e7fc9a6e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:25.969515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde88f3-8006-4c6c-96dd-50e7fc9a6e52 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:25.969873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde88f3-8006-4c6c-96dd-50e7fc9a6e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:25.970226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde88f3-8006-4c6c-96dd-50e7fc9a6e52 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:25.971323 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde88f3-8006-4c6c-96dd-50e7fc9a6e52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:25.977101 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde88f3-8006-4c6c-96dd-50e7fc9a6e52 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:25.977192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:26.007093 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92ac08-523c-4482-b78c-3311fe14e4df None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:26.059894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1095/2187] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:14:25 2022] POST /identity/v3/users => generated 288 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:26.064175 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cde88f3-8006-4c6c-96dd-50e7fc9a6e52 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:26.069611 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011ba15-acde-4605-ac89-695d7252a14b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:26.070248 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011ba15-acde-4605-ac89-695d7252a14b service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:26.070582 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011ba15-acde-4605-ac89-695d7252a14b service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:26.070891 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011ba15-acde-4605-ac89-695d7252a14b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:26.075838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cde88f3-8006-4c6c-96dd-50e7fc9a6e52 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:26.080881 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdffe089-1706-4b7c-a7ca-3c9c2d539419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:26.081494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdffe089-1706-4b7c-a7ca-3c9c2d539419 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:26.081778 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdffe089-1706-4b7c-a7ca-3c9c2d539419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:26.082226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdffe089-1706-4b7c-a7ca-3c9c2d539419 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:26.082959 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdffe089-1706-4b7c-a7ca-3c9c2d539419 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:26.084580 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdffe089-1706-4b7c-a7ca-3c9c2d539419 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:26.096753 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1096/2188] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:14:26.100827 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdffe089-1706-4b7c-a7ca-3c9c2d539419 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:26.111544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdffe089-1706-4b7c-a7ca-3c9c2d539419 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:26.116764 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae582f85-4ff2-4ef7-b474-d4e3019f979a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:26.117376 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae582f85-4ff2-4ef7-b474-d4e3019f979a admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:26.117980 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae582f85-4ff2-4ef7-b474-d4e3019f979a admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:26.118262 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae582f85-4ff2-4ef7-b474-d4e3019f979a admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:26.119202 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae582f85-4ff2-4ef7-b474-d4e3019f979a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:26.120799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae582f85-4ff2-4ef7-b474-d4e3019f979a admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:26.131667 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1097/2189] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:14:26.135872 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae582f85-4ff2-4ef7-b474-d4e3019f979a admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:26.136543 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011ba15-acde-4605-ac89-695d7252a14b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:26.138139 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011ba15-acde-4605-ac89-695d7252a14b service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:26.145797 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae582f85-4ff2-4ef7-b474-d4e3019f979a admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:26.149561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1093/2190] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:14:25 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:14:26.151145 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acbf6e9-95d3-429c-b627-e9331e0806ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:26.152034 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbf6e9-95d3-429c-b627-e9331e0806ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:26.152372 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbf6e9-95d3-429c-b627-e9331e0806ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:26.152660 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbf6e9-95d3-429c-b627-e9331e0806ee admin admin] PATH_INFO: `/v3/projects/2a9a04efa8e144e48b8a671c51fdaa5e/users/6203acf97abc4fb18f8220f8ed34afc1/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:26.170039 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acbf6e9-95d3-429c-b627-e9331e0806ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a9a04efa8e144e48b8a671c51fdaa5e, user_id=6203acf97abc4fb18f8220f8ed34afc1, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:26.173836 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acbf6e9-95d3-429c-b627-e9331e0806ee admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:26.280584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1098/2191] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:14:26 2022] PUT /identity/v3/projects/2a9a04efa8e144e48b8a671c51fdaa5e/users/6203acf97abc4fb18f8220f8ed34afc1/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:14:26.290131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011ba15-acde-4605-ac89-695d7252a14b service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:26.292256 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23743fba-2b2a-459b-adc5-b02a06b46c57 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:26.292553 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23743fba-2b2a-459b-adc5-b02a06b46c57 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:26.292837 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23743fba-2b2a-459b-adc5-b02a06b46c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:26.302093 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011ba15-acde-4605-ac89-695d7252a14b service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:26.323329 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-23743fba-2b2a-459b-adc5-b02a06b46c57 None None] MFA Rules not processed for user `6203acf97abc4fb18f8220f8ed34afc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:26.324681 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-23743fba-2b2a-459b-adc5-b02a06b46c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:26.350607 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:26.362260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf2350-008b-462b-9f61-629c492123f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:26.362965 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf2350-008b-462b-9f61-629c492123f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:26.363251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf2350-008b-462b-9f61-629c492123f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:26.363529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf2350-008b-462b-9f61-629c492123f1 admin admin] PATH_INFO: `/v3/users/f0606defb69a4badb994f764e0916802` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:26.364730 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf2350-008b-462b-9f61-629c492123f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0606defb69a4badb994f764e0916802)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:26.366699 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf2350-008b-462b-9f61-629c492123f1 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:26.393699 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1099/2192] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:26.404645 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23743fba-2b2a-459b-adc5-b02a06b46c57 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:26.459800 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23743fba-2b2a-459b-adc5-b02a06b46c57 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:26.463615 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bff4c1-051f-42ef-828c-ea9a3c512144 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:26.464234 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bff4c1-051f-42ef-828c-ea9a3c512144 service cinder] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:26.464521 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bff4c1-051f-42ef-828c-ea9a3c512144 service cinder] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:26.464805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bff4c1-051f-42ef-828c-ea9a3c512144 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:26.473701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bff4c1-051f-42ef-828c-ea9a3c512144 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:26.475379 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bff4c1-051f-42ef-828c-ea9a3c512144 service cinder] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:26.484944 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1100/2193] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:14:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:14:26.566828 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-dcbf2350-008b-462b-9f61-629c492123f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0606defb69a4badb994f764e0916802'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:26.660211 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-dcbf2350-008b-462b-9f61-629c492123f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0606defb69a4badb994f764e0916802'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:26.662142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:26.663316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-dcbf2350-008b-462b-9f61-629c492123f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0606defb69a4badb994f764e0916802'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:26.677349 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-dcbf2350-008b-462b-9f61-629c492123f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0606defb69a4badb994f764e0916802'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:26.678120 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-dcbf2350-008b-462b-9f61-629c492123f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0606defb69a4badb994f764e0916802'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:26.690703 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1094/2194] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:14:26 2022] DELETE /identity/v3/users/f0606defb69a4badb994f764e0916802 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:26.695383 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9760b-f9fb-4895-86e7-154cb1fedc7e None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:26.695671 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9760b-f9fb-4895-86e7-154cb1fedc7e None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:26.696003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9760b-f9fb-4895-86e7-154cb1fedc7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:26.715121 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-0df9760b-f9fb-4895-86e7-154cb1fedc7e None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:26.716329 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-0df9760b-f9fb-4895-86e7-154cb1fedc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:26.774685 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1101/2195] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:26.785154 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf2350-008b-462b-9f61-629c492123f1 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:26.786764 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-dcbf2350-008b-462b-9f61-629c492123f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:26.844726 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf2350-008b-462b-9f61-629c492123f1 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:26.848670 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d9506-c940-4ac2-898b-bc69ce82177c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:26.849280 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d9506-c940-4ac2-898b-bc69ce82177c service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:26.849603 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d9506-c940-4ac2-898b-bc69ce82177c service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:26.849889 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d9506-c940-4ac2-898b-bc69ce82177c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:26.858705 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0d9506-c940-4ac2-898b-bc69ce82177c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:26.860750 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0d9506-c940-4ac2-898b-bc69ce82177c service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:26.870183 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1095/2196] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:14:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:14:26.996679 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:27.454771 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df9760b-f9fb-4895-86e7-154cb1fedc7e None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:27.456644 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-0df9760b-f9fb-4895-86e7-154cb1fedc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:27.481634 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df9760b-f9fb-4895-86e7-154cb1fedc7e None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:27.493231 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502b411-1bd1-4827-a9ba-dc400fbf7132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:27.494333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502b411-1bd1-4827-a9ba-dc400fbf7132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:27.494626 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502b411-1bd1-4827-a9ba-dc400fbf7132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:27.494945 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502b411-1bd1-4827-a9ba-dc400fbf7132 admin admin] PATH_INFO: `/v3/projects/b3fa064fa97f4831b167b339a52ea987` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:27.497265 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a502b411-1bd1-4827-a9ba-dc400fbf7132 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3fa064fa97f4831b167b339a52ea987)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:27.499612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a502b411-1bd1-4827-a9ba-dc400fbf7132 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:27.532740 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:27.619596 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d9506-c940-4ac2-898b-bc69ce82177c service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:27.621199 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ec0d9506-c940-4ac2-898b-bc69ce82177c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:27.650558 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a502b411-1bd1-4827-a9ba-dc400fbf7132 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3fa064fa97f4831b167b339a52ea987', 'request_id': 'req-a502b411-1bd1-4827-a9ba-dc400fbf7132'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:27.657607 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a502b411-1bd1-4827-a9ba-dc400fbf7132 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3fa064fa97f4831b167b339a52ea987', 'request_id': 'req-a502b411-1bd1-4827-a9ba-dc400fbf7132'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:27.665623 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:27.698600 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d9506-c940-4ac2-898b-bc69ce82177c service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:27.749132 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ad085-2ea3-4a56-8ea2-c3bf38764942 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:27.751021 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ad085-2ea3-4a56-8ea2-c3bf38764942 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:27.751021 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ad085-2ea3-4a56-8ea2-c3bf38764942 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:27.751021 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ad085-2ea3-4a56-8ea2-c3bf38764942 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:27.751343 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a502b411-1bd1-4827-a9ba-dc400fbf7132 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3fa064fa97f4831b167b339a52ea987', 'request_id': 'req-a502b411-1bd1-4827-a9ba-dc400fbf7132'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:27.752808 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-003ad085-2ea3-4a56-8ea2-c3bf38764942 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:27.762661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a502b411-1bd1-4827-a9ba-dc400fbf7132 admin admin] The token cache is being invalidate because project b3fa064fa97f4831b167b339a52ea987 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:14:27.762988 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a502b411-1bd1-4827-a9ba-dc400fbf7132 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:27.763647 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a502b411-1bd1-4827-a9ba-dc400fbf7132 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:27.765855 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1102/2197] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:14:27 2022] DELETE /identity/v3/projects/b3fa064fa97f4831b167b339a52ea987 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:27.769735 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502b411-1bd1-4827-a9ba-dc400fbf7132 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:27.775680 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-a502b411-1bd1-4827-a9ba-dc400fbf7132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:27.813037 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ad085-2ea3-4a56-8ea2-c3bf38764942 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:27.814479 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ad085-2ea3-4a56-8ea2-c3bf38764942 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:27.824997 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1096/2198] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:14:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:14:27.844330 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502b411-1bd1-4827-a9ba-dc400fbf7132 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:27.849738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130179d2-bdb6-43f2-9a48-b48db4b2a898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:27.850725 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130179d2-bdb6-43f2-9a48-b48db4b2a898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:27.851008 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130179d2-bdb6-43f2-9a48-b48db4b2a898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:27.851400 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130179d2-bdb6-43f2-9a48-b48db4b2a898 admin admin] PATH_INFO: `/v3/users/331b2ee0c8f74548a206d50cadd4f813` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:27.852993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130179d2-bdb6-43f2-9a48-b48db4b2a898 admin admin] RBAC: Authorizing `identity:delete_user(user_id=331b2ee0c8f74548a206d50cadd4f813)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:27.855330 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130179d2-bdb6-43f2-9a48-b48db4b2a898 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:28.073313 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-130179d2-bdb6-43f2-9a48-b48db4b2a898 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '331b2ee0c8f74548a206d50cadd4f813'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:28.160581 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-130179d2-bdb6-43f2-9a48-b48db4b2a898 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '331b2ee0c8f74548a206d50cadd4f813'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:28.160687 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-130179d2-bdb6-43f2-9a48-b48db4b2a898 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '331b2ee0c8f74548a206d50cadd4f813'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:28.174267 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-130179d2-bdb6-43f2-9a48-b48db4b2a898 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '331b2ee0c8f74548a206d50cadd4f813'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:28.175027 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-130179d2-bdb6-43f2-9a48-b48db4b2a898 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '331b2ee0c8f74548a206d50cadd4f813'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:28.189640 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1103/2199] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:14:27 2022] DELETE /identity/v3/users/331b2ee0c8f74548a206d50cadd4f813 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:14:28.736951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ad085-2ea3-4a56-8ea2-c3bf38764942 service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:28.747655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ad085-2ea3-4a56-8ea2-c3bf38764942 service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:28.814927 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a3803-fbe4-42a2-9903-35c7526e7118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:28.815731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a3803-fbe4-42a2-9903-35c7526e7118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:28.815878 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a3803-fbe4-42a2-9903-35c7526e7118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:28.816116 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a3803-fbe4-42a2-9903-35c7526e7118 admin admin] PATH_INFO: `/v3/projects/cd947cc5313444e2b22b0c56cc55e812` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:28.817286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9a3803-fbe4-42a2-9903-35c7526e7118 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd947cc5313444e2b22b0c56cc55e812)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:28.819237 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9a3803-fbe4-42a2-9903-35c7526e7118 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:28.933997 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:28.943785 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2e9a3803-fbe4-42a2-9903-35c7526e7118 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd947cc5313444e2b22b0c56cc55e812', 'request_id': 'req-2e9a3803-fbe4-42a2-9903-35c7526e7118'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:29.036301 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2e9a3803-fbe4-42a2-9903-35c7526e7118 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd947cc5313444e2b22b0c56cc55e812', 'request_id': 'req-2e9a3803-fbe4-42a2-9903-35c7526e7118'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:29.042774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2e9a3803-fbe4-42a2-9903-35c7526e7118 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd947cc5313444e2b22b0c56cc55e812', 'request_id': 'req-2e9a3803-fbe4-42a2-9903-35c7526e7118'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:29.055577 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2e9a3803-fbe4-42a2-9903-35c7526e7118 admin admin] The token cache is being invalidate because project cd947cc5313444e2b22b0c56cc55e812 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:14:29.055874 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2e9a3803-fbe4-42a2-9903-35c7526e7118 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:29.056537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2e9a3803-fbe4-42a2-9903-35c7526e7118 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:29.058141 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1097/2200] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:14:28 2022] DELETE /identity/v3/projects/cd947cc5313444e2b22b0c56cc55e812 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:29.063203 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf49ea-fb3c-45f1-a7a6-5df2a41e6de7 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:29.063425 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf49ea-fb3c-45f1-a7a6-5df2a41e6de7 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:29.063713 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf49ea-fb3c-45f1-a7a6-5df2a41e6de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:29.083398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-a6cf49ea-fb3c-45f1-a7a6-5df2a41e6de7 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:29.084617 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-a6cf49ea-fb3c-45f1-a7a6-5df2a41e6de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:29.105762 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:29.152527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1104/2201] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:29.161295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e28bd-5fcb-4eca-84a2-496605569e15 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:29.161587 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e28bd-5fcb-4eca-84a2-496605569e15 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:29.161866 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e28bd-5fcb-4eca-84a2-496605569e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:29.183496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-b63e28bd-5fcb-4eca-84a2-496605569e15 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:29.184467 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b63e28bd-5fcb-4eca-84a2-496605569e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:29.192256 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1098/2202] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:29.200729 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf49ea-fb3c-45f1-a7a6-5df2a41e6de7 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:29.211089 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf49ea-fb3c-45f1-a7a6-5df2a41e6de7 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:29.217096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba44f0e-aac5-45e5-8e64-00c8b7ab29fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:29.217768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba44f0e-aac5-45e5-8e64-00c8b7ab29fe admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:29.218017 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba44f0e-aac5-45e5-8e64-00c8b7ab29fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:29.218984 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba44f0e-aac5-45e5-8e64-00c8b7ab29fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:29.219702 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba44f0e-aac5-45e5-8e64-00c8b7ab29fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:14:29.220012 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba44f0e-aac5-45e5-8e64-00c8b7ab29fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:29.221932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba44f0e-aac5-45e5-8e64-00c8b7ab29fe admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:29.238038 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1105/2203] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:14:29.243202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e28bd-5fcb-4eca-84a2-496605569e15 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:29.253595 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e28bd-5fcb-4eca-84a2-496605569e15 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:29.259082 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f1b65-b89a-4632-abd3-4f5b8566704a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:29.259767 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f1b65-b89a-4632-abd3-4f5b8566704a admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:29.260060 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f1b65-b89a-4632-abd3-4f5b8566704a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:29.260340 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f1b65-b89a-4632-abd3-4f5b8566704a admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:29.264310 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f1b65-b89a-4632-abd3-4f5b8566704a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:29.264310 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f1b65-b89a-4632-abd3-4f5b8566704a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:29.395950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1099/2204] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:14:29 2022] POST /identity/v3/projects => generated 377 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:14:29.400256 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba44f0e-aac5-45e5-8e64-00c8b7ab29fe admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:29.410429 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba44f0e-aac5-45e5-8e64-00c8b7ab29fe admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:29.469825 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648dbf5a-e061-4b9e-bf62-e18d473bf0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:29.470450 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648dbf5a-e061-4b9e-bf62-e18d473bf0a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:29.471046 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648dbf5a-e061-4b9e-bf62-e18d473bf0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:29.471326 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648dbf5a-e061-4b9e-bf62-e18d473bf0a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:29.472003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648dbf5a-e061-4b9e-bf62-e18d473bf0a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:29.474113 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648dbf5a-e061-4b9e-bf62-e18d473bf0a7 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:29.589381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1106/2205] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:14:29 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:14:29.595191 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f1b65-b89a-4632-abd3-4f5b8566704a admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:29.606574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f1b65-b89a-4632-abd3-4f5b8566704a admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:29.613501 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d5954c-a141-42d3-ab13-fa6b6e3bc979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:29.614127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5954c-a141-42d3-ab13-fa6b6e3bc979 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:29.614563 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5954c-a141-42d3-ab13-fa6b6e3bc979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:29.618541 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5954c-a141-42d3-ab13-fa6b6e3bc979 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:29.618541 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d5954c-a141-42d3-ab13-fa6b6e3bc979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:29.618541 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d5954c-a141-42d3-ab13-fa6b6e3bc979 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:29.632637 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1100/2206] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:14:29.633972 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648dbf5a-e061-4b9e-bf62-e18d473bf0a7 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:29.644632 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648dbf5a-e061-4b9e-bf62-e18d473bf0a7 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:29.649757 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e12995-e0f2-467b-b067-875b61945ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:29.650542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e12995-e0f2-467b-b067-875b61945ef9 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:29.650758 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e12995-e0f2-467b-b067-875b61945ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:29.651071 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e12995-e0f2-467b-b067-875b61945ef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:29.651819 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e12995-e0f2-467b-b067-875b61945ef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:29.653458 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e12995-e0f2-467b-b067-875b61945ef9 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:29.665489 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1107/2207] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:14:29.673878 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d5954c-a141-42d3-ab13-fa6b6e3bc979 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:29.684857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d5954c-a141-42d3-ab13-fa6b6e3bc979 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:29.691423 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a210c-f624-407e-9adb-1918ed4cd719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:29.692086 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a210c-f624-407e-9adb-1918ed4cd719 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:29.692377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a210c-f624-407e-9adb-1918ed4cd719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:29.692662 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a210c-f624-407e-9adb-1918ed4cd719 admin admin] PATH_INFO: `/v3/projects/ef8af2a50aeb4f8689a5df42249a7fa2/users/e65ca94acd264843a2d1748c18489473/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:29.708135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0a210c-f624-407e-9adb-1918ed4cd719 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef8af2a50aeb4f8689a5df42249a7fa2, user_id=e65ca94acd264843a2d1748c18489473, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:29.710982 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0a210c-f624-407e-9adb-1918ed4cd719 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:29.772625 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1101/2208] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:14:29 2022] PUT /identity/v3/projects/ef8af2a50aeb4f8689a5df42249a7fa2/users/e65ca94acd264843a2d1748c18489473/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:29.783012 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b2a4b-8fde-4bf9-8a30-1db2400d0e8c None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:29.783306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b2a4b-8fde-4bf9-8a30-1db2400d0e8c None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:29.783590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b2a4b-8fde-4bf9-8a30-1db2400d0e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:29.814348 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-348b2a4b-8fde-4bf9-8a30-1db2400d0e8c None None] MFA Rules not processed for user `e65ca94acd264843a2d1748c18489473`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:29.815598 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-348b2a4b-8fde-4bf9-8a30-1db2400d0e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:29.895971 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a210c-f624-407e-9adb-1918ed4cd719 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:29.896682 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1108/2209] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:29.898120 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ae0a210c-f624-407e-9adb-1918ed4cd719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:29.901242 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b2a4b-8fde-4bf9-8a30-1db2400d0e8c None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:29.930741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:29.976820 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a210c-f624-407e-9adb-1918ed4cd719 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:29.977202 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b2a4b-8fde-4bf9-8a30-1db2400d0e8c None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:29.981035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442020e2-68f2-4e40-8216-fdab1476d987 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:29.981694 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442020e2-68f2-4e40-8216-fdab1476d987 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:29.986544 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442020e2-68f2-4e40-8216-fdab1476d987 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:29.986544 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442020e2-68f2-4e40-8216-fdab1476d987 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:29.986758 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b906825-1e87-41fd-a0ce-5fe8bf27c31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:29.986758 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b906825-1e87-41fd-a0ce-5fe8bf27c31d admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:29.986758 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b906825-1e87-41fd-a0ce-5fe8bf27c31d admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:29.986758 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b906825-1e87-41fd-a0ce-5fe8bf27c31d admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:29.987111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b906825-1e87-41fd-a0ce-5fe8bf27c31d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:29.987193 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-442020e2-68f2-4e40-8216-fdab1476d987 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:29.987280 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b906825-1e87-41fd-a0ce-5fe8bf27c31d admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:30.059122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442020e2-68f2-4e40-8216-fdab1476d987 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:30.060588 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442020e2-68f2-4e40-8216-fdab1476d987 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:30.073188 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1109/2210] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:14:29 2022] POST /identity/v3/projects => generated 379 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:30.073595 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1102/2211] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:14:29 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:14:30.077360 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b906825-1e87-41fd-a0ce-5fe8bf27c31d admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:30.147783 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b906825-1e87-41fd-a0ce-5fe8bf27c31d admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:30.158584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4706697a-de40-4873-a3de-4d8258a824d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:30.158584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706697a-de40-4873-a3de-4d8258a824d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:30.158584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706697a-de40-4873-a3de-4d8258a824d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:30.158584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706697a-de40-4873-a3de-4d8258a824d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:30.159134 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4706697a-de40-4873-a3de-4d8258a824d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:30.159134 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4706697a-de40-4873-a3de-4d8258a824d5 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:30.291722 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1110/2212] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:14:30 2022] POST /identity/v3/users => generated 290 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:14:30.295815 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442020e2-68f2-4e40-8216-fdab1476d987 service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:30.305954 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442020e2-68f2-4e40-8216-fdab1476d987 service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:30.310557 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2788e0-7668-4c1a-9e12-8c48bfd305d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:30.311188 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2788e0-7668-4c1a-9e12-8c48bfd305d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:30.311470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2788e0-7668-4c1a-9e12-8c48bfd305d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:30.311744 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2788e0-7668-4c1a-9e12-8c48bfd305d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:30.312550 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2788e0-7668-4c1a-9e12-8c48bfd305d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:30.314157 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2788e0-7668-4c1a-9e12-8c48bfd305d7 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:30.316720 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:30.325436 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1103/2213] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:14:30.329892 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4706697a-de40-4873-a3de-4d8258a824d5 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:30.340540 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4706697a-de40-4873-a3de-4d8258a824d5 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:30.346319 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47de8bb-9751-4cff-bf39-c45e860742c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:30.353882 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47de8bb-9751-4cff-bf39-c45e860742c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:30.353882 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47de8bb-9751-4cff-bf39-c45e860742c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:30.353882 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47de8bb-9751-4cff-bf39-c45e860742c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:30.353882 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47de8bb-9751-4cff-bf39-c45e860742c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:30.353882 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47de8bb-9751-4cff-bf39-c45e860742c7 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:30.360494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1111/2214] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:14:30.368122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2788e0-7668-4c1a-9e12-8c48bfd305d7 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:30.378446 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2788e0-7668-4c1a-9e12-8c48bfd305d7 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:30.383196 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3979c7e-f68f-4444-b865-35909a5abeb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:30.383817 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3979c7e-f68f-4444-b865-35909a5abeb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:30.384104 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3979c7e-f68f-4444-b865-35909a5abeb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:30.384386 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3979c7e-f68f-4444-b865-35909a5abeb1 admin admin] PATH_INFO: `/v3/projects/9b4624a35d8142d184ff323dfdd85d9f/users/c4005f979a78402eb89a9dc357f423c9/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:30.400274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3979c7e-f68f-4444-b865-35909a5abeb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b4624a35d8142d184ff323dfdd85d9f, user_id=c4005f979a78402eb89a9dc357f423c9, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:30.403021 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3979c7e-f68f-4444-b865-35909a5abeb1 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:30.487662 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1104/2215] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:14:30 2022] PUT /identity/v3/projects/9b4624a35d8142d184ff323dfdd85d9f/users/c4005f979a78402eb89a9dc357f423c9/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:14:30.497789 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e2bf7-5c2e-47fc-b14a-1e75f7af28c0 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:30.498084 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e2bf7-5c2e-47fc-b14a-1e75f7af28c0 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:30.498365 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e2bf7-5c2e-47fc-b14a-1e75f7af28c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:30.527144 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-e64e2bf7-5c2e-47fc-b14a-1e75f7af28c0 None None] MFA Rules not processed for user `c4005f979a78402eb89a9dc357f423c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:30.528364 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-e64e2bf7-5c2e-47fc-b14a-1e75f7af28c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:30.592221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1112/2216] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:30.597089 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3979c7e-f68f-4444-b865-35909a5abeb1 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:30.660250 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3979c7e-f68f-4444-b865-35909a5abeb1 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:30.664994 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee81f2-5487-42dc-a0ed-878e10f47aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:30.665618 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee81f2-5487-42dc-a0ed-878e10f47aca admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:30.665904 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee81f2-5487-42dc-a0ed-878e10f47aca admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:30.666187 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee81f2-5487-42dc-a0ed-878e10f47aca admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:30.666940 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee81f2-5487-42dc-a0ed-878e10f47aca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:30.668897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee81f2-5487-42dc-a0ed-878e10f47aca admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:30.869827 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1105/2217] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:14:30 2022] POST /identity/v3/projects => generated 379 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:30.871433 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e2bf7-5c2e-47fc-b14a-1e75f7af28c0 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:30.884546 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e2bf7-5c2e-47fc-b14a-1e75f7af28c0 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:30.900011 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:30.947733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66fa79-50b3-4d7d-881a-687435767959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:30.948355 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66fa79-50b3-4d7d-881a-687435767959 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:30.948638 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66fa79-50b3-4d7d-881a-687435767959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:30.948916 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66fa79-50b3-4d7d-881a-687435767959 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:30.949580 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e66fa79-50b3-4d7d-881a-687435767959 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:30.951599 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e66fa79-50b3-4d7d-881a-687435767959 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:31.039829 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1113/2218] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:14:30 2022] POST /identity/v3/users => generated 296 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:31.044179 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee81f2-5487-42dc-a0ed-878e10f47aca admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:31.055760 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee81f2-5487-42dc-a0ed-878e10f47aca admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:31.068278 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e7cff-6581-484f-a807-a1826c73f12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:31.068904 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e7cff-6581-484f-a807-a1826c73f12a admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:31.069186 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e7cff-6581-484f-a807-a1826c73f12a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:31.069461 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e7cff-6581-484f-a807-a1826c73f12a admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:31.070161 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9e7cff-6581-484f-a807-a1826c73f12a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:31.071874 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9e7cff-6581-484f-a807-a1826c73f12a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:31.083189 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1106/2219] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:14:31.087625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66fa79-50b3-4d7d-881a-687435767959 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:31.098306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66fa79-50b3-4d7d-881a-687435767959 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:31.103126 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c853b6d-64bb-4a38-bbaa-d4d80abb6183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:31.103787 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c853b6d-64bb-4a38-bbaa-d4d80abb6183 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:31.104072 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c853b6d-64bb-4a38-bbaa-d4d80abb6183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:31.104391 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c853b6d-64bb-4a38-bbaa-d4d80abb6183 admin admin] PATH_INFO: `/v3/projects/9c362a83db8e4eacaf885de9779eba7a/users/7db6a56d86ec4c309acbe16922d46035/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:31.119826 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c853b6d-64bb-4a38-bbaa-d4d80abb6183 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c362a83db8e4eacaf885de9779eba7a, user_id=7db6a56d86ec4c309acbe16922d46035, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:31.125190 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c853b6d-64bb-4a38-bbaa-d4d80abb6183 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:31.237350 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1114/2220] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:14:31 2022] PUT /identity/v3/projects/9c362a83db8e4eacaf885de9779eba7a/users/7db6a56d86ec4c309acbe16922d46035/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:14:31.241808 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e7cff-6581-484f-a807-a1826c73f12a admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:31.252532 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e7cff-6581-484f-a807-a1826c73f12a admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:31.325847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34cf3db-7bf2-4c04-9b44-0a420ce90a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:31.327046 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cf3db-7bf2-4c04-9b44-0a420ce90a25 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:31.327611 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cf3db-7bf2-4c04-9b44-0a420ce90a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:31.328640 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cf3db-7bf2-4c04-9b44-0a420ce90a25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:31.330027 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34cf3db-7bf2-4c04-9b44-0a420ce90a25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:31.333364 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34cf3db-7bf2-4c04-9b44-0a420ce90a25 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:31.347862 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1107/2221] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:14:31.352031 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c853b6d-64bb-4a38-bbaa-d4d80abb6183 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:31.363224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c853b6d-64bb-4a38-bbaa-d4d80abb6183 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:31.368424 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1931caa-d0b6-456a-ad2e-ce7256c825d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:31.369091 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1931caa-d0b6-456a-ad2e-ce7256c825d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:31.369374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1931caa-d0b6-456a-ad2e-ce7256c825d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:31.369655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1931caa-d0b6-456a-ad2e-ce7256c825d3 admin admin] PATH_INFO: `/v3/projects/9c362a83db8e4eacaf885de9779eba7a/users/7db6a56d86ec4c309acbe16922d46035/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:31.371789 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1931caa-d0b6-456a-ad2e-ce7256c825d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c362a83db8e4eacaf885de9779eba7a, user_id=7db6a56d86ec4c309acbe16922d46035, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:31.374472 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1931caa-d0b6-456a-ad2e-ce7256c825d3 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:31.387623 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1115/2222] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:14:31 2022] PUT /identity/v3/projects/9c362a83db8e4eacaf885de9779eba7a/users/7db6a56d86ec4c309acbe16922d46035/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:14:31.399088 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347edaa-0f2f-426f-9fbd-2cb846cd31f1 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:31.399385 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347edaa-0f2f-426f-9fbd-2cb846cd31f1 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:31.399668 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347edaa-0f2f-426f-9fbd-2cb846cd31f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:31.437831 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-4347edaa-0f2f-426f-9fbd-2cb846cd31f1 None None] MFA Rules not processed for user `7db6a56d86ec4c309acbe16922d46035`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:31.439299 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-4347edaa-0f2f-426f-9fbd-2cb846cd31f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:31.474268 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:31.510878 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1108/2223] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:31 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:31.523220 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1931caa-d0b6-456a-ad2e-ce7256c825d3 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:31.525110 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-a1931caa-d0b6-456a-ad2e-ce7256c825d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:31.583910 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1931caa-d0b6-456a-ad2e-ce7256c825d3 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:31.587523 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40516b90-96bf-402b-b4e6-e79eaa1480a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:31.588140 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40516b90-96bf-402b-b4e6-e79eaa1480a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:31.588423 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40516b90-96bf-402b-b4e6-e79eaa1480a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:31.588703 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40516b90-96bf-402b-b4e6-e79eaa1480a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:31.598706 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40516b90-96bf-402b-b4e6-e79eaa1480a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:31.600189 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40516b90-96bf-402b-b4e6-e79eaa1480a4 service cinder] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:31.609762 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1116/2224] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:14:31 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:14:31.721823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347edaa-0f2f-426f-9fbd-2cb846cd31f1 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:31.731624 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347edaa-0f2f-426f-9fbd-2cb846cd31f1 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:31.735212 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd3105-afb0-4bdf-8d94-6fe7c9fdb7cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:31.735975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd3105-afb0-4bdf-8d94-6fe7c9fdb7cd service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:31.736332 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd3105-afb0-4bdf-8d94-6fe7c9fdb7cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:31.736612 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd3105-afb0-4bdf-8d94-6fe7c9fdb7cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:31.796394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd3105-afb0-4bdf-8d94-6fe7c9fdb7cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:31.797881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd3105-afb0-4bdf-8d94-6fe7c9fdb7cd service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:31.807071 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1109/2225] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:14:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:14:33.657015 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2e664-f4d0-424e-aebf-457f131754ae None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:33.657940 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2e664-f4d0-424e-aebf-457f131754ae None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:33.658374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2e664-f4d0-424e-aebf-457f131754ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:33.677939 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-55d2e664-f4d0-424e-aebf-457f131754ae None None] MFA Rules not processed for user `8cbb45074cef4d5082580c4c5130eb9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:33.679180 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-55d2e664-f4d0-424e-aebf-457f131754ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:33.738529 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1117/2226] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:36.338614 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd3105-afb0-4bdf-8d94-6fe7c9fdb7cd service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:36.457931 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd3105-afb0-4bdf-8d94-6fe7c9fdb7cd service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:36.465097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e551e-baed-4c60-8050-2872bc825782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:36.466165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e551e-baed-4c60-8050-2872bc825782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:36.466685 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e551e-baed-4c60-8050-2872bc825782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:36.467142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e551e-baed-4c60-8050-2872bc825782 admin admin] PATH_INFO: `/v3/users/e65ca94acd264843a2d1748c18489473` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:36.469252 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e551e-baed-4c60-8050-2872bc825782 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e65ca94acd264843a2d1748c18489473)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:36.472518 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e551e-baed-4c60-8050-2872bc825782 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:36.707055 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-538e551e-baed-4c60-8050-2872bc825782 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e65ca94acd264843a2d1748c18489473'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:36.799509 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-538e551e-baed-4c60-8050-2872bc825782 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e65ca94acd264843a2d1748c18489473'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:36.800143 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-538e551e-baed-4c60-8050-2872bc825782 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e65ca94acd264843a2d1748c18489473'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:36.812096 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:36.813359 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-538e551e-baed-4c60-8050-2872bc825782 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e65ca94acd264843a2d1748c18489473'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:36.814069 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-538e551e-baed-4c60-8050-2872bc825782 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e65ca94acd264843a2d1748c18489473'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:36.826182 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1110/2227] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:14:36 2022] DELETE /identity/v3/users/e65ca94acd264843a2d1748c18489473 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:36.834246 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33eaba8-ba21-4f03-b50a-acd502daeb08 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:36.835126 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33eaba8-ba21-4f03-b50a-acd502daeb08 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:36.835411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33eaba8-ba21-4f03-b50a-acd502daeb08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:36.856224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-d33eaba8-ba21-4f03-b50a-acd502daeb08 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:36.857484 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-d33eaba8-ba21-4f03-b50a-acd502daeb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:36.879132 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:36.977031 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1118/2228] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:14:36.988898 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e551e-baed-4c60-8050-2872bc825782 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:36.990647 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-538e551e-baed-4c60-8050-2872bc825782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:37.059735 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33eaba8-ba21-4f03-b50a-acd502daeb08 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:37.061587 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-d33eaba8-ba21-4f03-b50a-acd502daeb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:37.086434 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33eaba8-ba21-4f03-b50a-acd502daeb08 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:37.104459 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037713f7-e2e6-473a-ac53-5e38a24ee6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:37.105137 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037713f7-e2e6-473a-ac53-5e38a24ee6e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:37.105418 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037713f7-e2e6-473a-ac53-5e38a24ee6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:37.106314 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037713f7-e2e6-473a-ac53-5e38a24ee6e9 admin admin] PATH_INFO: `/v3/users/0f97649f8a924b68b3c46e1723c328eb` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:37.109904 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037713f7-e2e6-473a-ac53-5e38a24ee6e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f97649f8a924b68b3c46e1723c328eb)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:37.111946 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037713f7-e2e6-473a-ac53-5e38a24ee6e9 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:37.113197 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e551e-baed-4c60-8050-2872bc825782 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:37.119881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d467d-6af5-4038-acab-15b2a695e3c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:37.121768 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d467d-6af5-4038-acab-15b2a695e3c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:37.122078 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d467d-6af5-4038-acab-15b2a695e3c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:37.122504 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d467d-6af5-4038-acab-15b2a695e3c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:37.137157 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d467d-6af5-4038-acab-15b2a695e3c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:37.138743 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d467d-6af5-4038-acab-15b2a695e3c1 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:37.156070 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1111/2229] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:14:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:14:37.434502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-037713f7-e2e6-473a-ac53-5e38a24ee6e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f97649f8a924b68b3c46e1723c328eb'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:37.525331 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-037713f7-e2e6-473a-ac53-5e38a24ee6e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f97649f8a924b68b3c46e1723c328eb'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:37.527742 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-037713f7-e2e6-473a-ac53-5e38a24ee6e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f97649f8a924b68b3c46e1723c328eb'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:37.576466 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-037713f7-e2e6-473a-ac53-5e38a24ee6e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f97649f8a924b68b3c46e1723c328eb'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:37.579182 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-037713f7-e2e6-473a-ac53-5e38a24ee6e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f97649f8a924b68b3c46e1723c328eb'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:37.602637 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1119/2230] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:14:37 2022] DELETE /identity/v3/users/0f97649f8a924b68b3c46e1723c328eb => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:14:37.607733 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5892a88-ec77-48a2-8507-fd2f6198a8e6 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:37.608234 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5892a88-ec77-48a2-8507-fd2f6198a8e6 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:37.608768 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5892a88-ec77-48a2-8507-fd2f6198a8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:37.632380 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-e5892a88-ec77-48a2-8507-fd2f6198a8e6 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:37.637270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e5892a88-ec77-48a2-8507-fd2f6198a8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:37.648542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037713f7-e2e6-473a-ac53-5e38a24ee6e9 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:37.650144 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-037713f7-e2e6-473a-ac53-5e38a24ee6e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:37.743142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1112/2231] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:37.746245 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037713f7-e2e6-473a-ac53-5e38a24ee6e9 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:37.751833 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba967c9-c4de-408f-9c46-72418a4ecbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:37.752571 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba967c9-c4de-408f-9c46-72418a4ecbcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:37.752854 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba967c9-c4de-408f-9c46-72418a4ecbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:37.753134 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba967c9-c4de-408f-9c46-72418a4ecbcd admin admin] PATH_INFO: `/v3/projects/ef8af2a50aeb4f8689a5df42249a7fa2` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:37.754319 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba967c9-c4de-408f-9c46-72418a4ecbcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef8af2a50aeb4f8689a5df42249a7fa2)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:37.755153 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5892a88-ec77-48a2-8507-fd2f6198a8e6 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:37.756479 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba967c9-c4de-408f-9c46-72418a4ecbcd admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:37.756749 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e5892a88-ec77-48a2-8507-fd2f6198a8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:37.825649 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5892a88-ec77-48a2-8507-fd2f6198a8e6 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:37.830053 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1298d-1b1b-4c63-8931-d2644f2c491e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:37.830754 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1298d-1b1b-4c63-8931-d2644f2c491e service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:37.831067 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1298d-1b1b-4c63-8931-d2644f2c491e service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:37.831344 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1298d-1b1b-4c63-8931-d2644f2c491e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:37.841365 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1298d-1b1b-4c63-8931-d2644f2c491e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:37.843416 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1298d-1b1b-4c63-8931-d2644f2c491e service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:37.852092 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:37.858416 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1113/2232] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:14:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:14:37.869015 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-3ba967c9-c4de-408f-9c46-72418a4ecbcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef8af2a50aeb4f8689a5df42249a7fa2', 'request_id': 'req-3ba967c9-c4de-408f-9c46-72418a4ecbcd'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:37.879905 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-3ba967c9-c4de-408f-9c46-72418a4ecbcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef8af2a50aeb4f8689a5df42249a7fa2', 'request_id': 'req-3ba967c9-c4de-408f-9c46-72418a4ecbcd'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:37.969183 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-3ba967c9-c4de-408f-9c46-72418a4ecbcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef8af2a50aeb4f8689a5df42249a7fa2', 'request_id': 'req-3ba967c9-c4de-408f-9c46-72418a4ecbcd'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:37.984238 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-3ba967c9-c4de-408f-9c46-72418a4ecbcd admin admin] The token cache is being invalidate because project ef8af2a50aeb4f8689a5df42249a7fa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:14:37.984536 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-3ba967c9-c4de-408f-9c46-72418a4ecbcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:37.985880 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-3ba967c9-c4de-408f-9c46-72418a4ecbcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:37.987509 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1120/2233] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:14:37 2022] DELETE /identity/v3/projects/ef8af2a50aeb4f8689a5df42249a7fa2 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:37.991510 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1298d-1b1b-4c63-8931-d2644f2c491e service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:38.002567 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1298d-1b1b-4c63-8931-d2644f2c491e service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:38.034632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:38.061912 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4f717-9e42-41af-ae3a-e024912cb9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:38.062464 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4f717-9e42-41af-ae3a-e024912cb9eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:38.062743 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4f717-9e42-41af-ae3a-e024912cb9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:38.063518 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4f717-9e42-41af-ae3a-e024912cb9eb admin admin] PATH_INFO: `/v3/users/c4005f979a78402eb89a9dc357f423c9` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:38.064511 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd4f717-9e42-41af-ae3a-e024912cb9eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4005f979a78402eb89a9dc357f423c9)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:38.067252 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd4f717-9e42-41af-ae3a-e024912cb9eb admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:38.305986 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-fcd4f717-9e42-41af-ae3a-e024912cb9eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4005f979a78402eb89a9dc357f423c9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:38.355127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba967c9-c4de-408f-9c46-72418a4ecbcd admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:38.356570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-3ba967c9-c4de-408f-9c46-72418a4ecbcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:38.370820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba967c9-c4de-408f-9c46-72418a4ecbcd admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:38.377534 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80d8c2-cf8f-4704-a4cc-e0e4beafa4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:38.378578 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80d8c2-cf8f-4704-a4cc-e0e4beafa4fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:38.378896 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80d8c2-cf8f-4704-a4cc-e0e4beafa4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:38.379198 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80d8c2-cf8f-4704-a4cc-e0e4beafa4fa admin admin] PATH_INFO: `/v3/projects/5121fbd6cff74504b78c2bfdd4fedf85` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:38.380625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80d8c2-cf8f-4704-a4cc-e0e4beafa4fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=5121fbd6cff74504b78c2bfdd4fedf85)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:38.382615 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80d8c2-cf8f-4704-a4cc-e0e4beafa4fa admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:38.416657 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-fcd4f717-9e42-41af-ae3a-e024912cb9eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4005f979a78402eb89a9dc357f423c9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:38.417567 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-fcd4f717-9e42-41af-ae3a-e024912cb9eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4005f979a78402eb89a9dc357f423c9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:38.432808 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-fcd4f717-9e42-41af-ae3a-e024912cb9eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4005f979a78402eb89a9dc357f423c9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:38.433522 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-fcd4f717-9e42-41af-ae3a-e024912cb9eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4005f979a78402eb89a9dc357f423c9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:38.447064 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1114/2234] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:14:37 2022] DELETE /identity/v3/users/c4005f979a78402eb89a9dc357f423c9 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:38.515661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-eb80d8c2-cf8f-4704-a4cc-e0e4beafa4fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5121fbd6cff74504b78c2bfdd4fedf85', 'request_id': 'req-eb80d8c2-cf8f-4704-a4cc-e0e4beafa4fa'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:38.522415 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-eb80d8c2-cf8f-4704-a4cc-e0e4beafa4fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5121fbd6cff74504b78c2bfdd4fedf85', 'request_id': 'req-eb80d8c2-cf8f-4704-a4cc-e0e4beafa4fa'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:38.601389 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-eb80d8c2-cf8f-4704-a4cc-e0e4beafa4fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5121fbd6cff74504b78c2bfdd4fedf85', 'request_id': 'req-eb80d8c2-cf8f-4704-a4cc-e0e4beafa4fa'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:38.613889 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-eb80d8c2-cf8f-4704-a4cc-e0e4beafa4fa admin admin] The token cache is being invalidate because project 5121fbd6cff74504b78c2bfdd4fedf85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:14:38.614186 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-eb80d8c2-cf8f-4704-a4cc-e0e4beafa4fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:38.614862 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-eb80d8c2-cf8f-4704-a4cc-e0e4beafa4fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:38.616529 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1121/2235] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:14:38 2022] DELETE /identity/v3/projects/5121fbd6cff74504b78c2bfdd4fedf85 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:38.623014 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4f717-9e42-41af-ae3a-e024912cb9eb admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:38.626792 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-fcd4f717-9e42-41af-ae3a-e024912cb9eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:38.719314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4f717-9e42-41af-ae3a-e024912cb9eb admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:38.797242 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c072ecd-741b-4e92-9990-3fbee27e0657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:38.797983 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c072ecd-741b-4e92-9990-3fbee27e0657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:38.798345 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c072ecd-741b-4e92-9990-3fbee27e0657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:38.798643 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c072ecd-741b-4e92-9990-3fbee27e0657 admin admin] PATH_INFO: `/v3/users/3ae4f616190147ef9ed4e19b6317006d` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:38.802116 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c072ecd-741b-4e92-9990-3fbee27e0657 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ae4f616190147ef9ed4e19b6317006d)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:38.804255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c072ecd-741b-4e92-9990-3fbee27e0657 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:38.806273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:38.941074 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80d8c2-cf8f-4704-a4cc-e0e4beafa4fa admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:38.944899 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-eb80d8c2-cf8f-4704-a4cc-e0e4beafa4fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:38.957454 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80d8c2-cf8f-4704-a4cc-e0e4beafa4fa admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:38.972906 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc291b-7a7b-47e7-88df-53b593ffe39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:38.972906 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc291b-7a7b-47e7-88df-53b593ffe39a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:38.972906 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc291b-7a7b-47e7-88df-53b593ffe39a admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:38.972906 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc291b-7a7b-47e7-88df-53b593ffe39a admin admin] PATH_INFO: `/v3/projects/9b4624a35d8142d184ff323dfdd85d9f` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:38.973598 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc291b-7a7b-47e7-88df-53b593ffe39a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b4624a35d8142d184ff323dfdd85d9f)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:38.973598 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc291b-7a7b-47e7-88df-53b593ffe39a admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:39.003352 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9c072ecd-741b-4e92-9990-3fbee27e0657 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ae4f616190147ef9ed4e19b6317006d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:39.098785 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9c072ecd-741b-4e92-9990-3fbee27e0657 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ae4f616190147ef9ed4e19b6317006d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:39.107273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9c072ecd-741b-4e92-9990-3fbee27e0657 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ae4f616190147ef9ed4e19b6317006d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:39.120968 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a1dc291b-7a7b-47e7-88df-53b593ffe39a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b4624a35d8142d184ff323dfdd85d9f', 'request_id': 'req-a1dc291b-7a7b-47e7-88df-53b593ffe39a'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:39.130095 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9c072ecd-741b-4e92-9990-3fbee27e0657 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ae4f616190147ef9ed4e19b6317006d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:39.130835 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9c072ecd-741b-4e92-9990-3fbee27e0657 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ae4f616190147ef9ed4e19b6317006d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:39.139269 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a1dc291b-7a7b-47e7-88df-53b593ffe39a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b4624a35d8142d184ff323dfdd85d9f', 'request_id': 'req-a1dc291b-7a7b-47e7-88df-53b593ffe39a'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:39.150182 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1115/2236] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:14:38 2022] DELETE /identity/v3/users/3ae4f616190147ef9ed4e19b6317006d => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:39.208716 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a1dc291b-7a7b-47e7-88df-53b593ffe39a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b4624a35d8142d184ff323dfdd85d9f', 'request_id': 'req-a1dc291b-7a7b-47e7-88df-53b593ffe39a'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:39.222560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a1dc291b-7a7b-47e7-88df-53b593ffe39a admin admin] The token cache is being invalidate because project 9b4624a35d8142d184ff323dfdd85d9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:14:39.222884 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a1dc291b-7a7b-47e7-88df-53b593ffe39a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:39.223569 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a1dc291b-7a7b-47e7-88df-53b593ffe39a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:39.225132 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1122/2237] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:14:38 2022] DELETE /identity/v3/projects/9b4624a35d8142d184ff323dfdd85d9f => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:39.229396 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c072ecd-741b-4e92-9990-3fbee27e0657 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:39.231501 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9c072ecd-741b-4e92-9990-3fbee27e0657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:39.305122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c072ecd-741b-4e92-9990-3fbee27e0657 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:39.310737 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70741d79-646e-4560-9f34-624a1da2d220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:39.311448 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70741d79-646e-4560-9f34-624a1da2d220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:39.311735 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70741d79-646e-4560-9f34-624a1da2d220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:39.312302 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70741d79-646e-4560-9f34-624a1da2d220 admin admin] PATH_INFO: `/v3/users/7db6a56d86ec4c309acbe16922d46035` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:39.313223 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70741d79-646e-4560-9f34-624a1da2d220 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7db6a56d86ec4c309acbe16922d46035)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:39.315198 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70741d79-646e-4560-9f34-624a1da2d220 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:39.517609 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-70741d79-646e-4560-9f34-624a1da2d220 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7db6a56d86ec4c309acbe16922d46035'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:39.576744 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-70741d79-646e-4560-9f34-624a1da2d220 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7db6a56d86ec4c309acbe16922d46035'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:39.578060 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-70741d79-646e-4560-9f34-624a1da2d220 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7db6a56d86ec4c309acbe16922d46035'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:39.591369 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-70741d79-646e-4560-9f34-624a1da2d220 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7db6a56d86ec4c309acbe16922d46035'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:39.592302 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-70741d79-646e-4560-9f34-624a1da2d220 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7db6a56d86ec4c309acbe16922d46035'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:39.605431 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1116/2238] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:14:39 2022] DELETE /identity/v3/users/7db6a56d86ec4c309acbe16922d46035 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:39.675778 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc291b-7a7b-47e7-88df-53b593ffe39a admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:39.678457 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-a1dc291b-7a7b-47e7-88df-53b593ffe39a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:39.714901 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=78,keystone:DELETE=19,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:39.758323 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc291b-7a7b-47e7-88df-53b593ffe39a admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:39.814300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c7b6a-7756-43b6-bc93-119a01d8fc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:39.814980 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c7b6a-7756-43b6-bc93-119a01d8fc83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:39.815267 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c7b6a-7756-43b6-bc93-119a01d8fc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:39.815544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c7b6a-7756-43b6-bc93-119a01d8fc83 admin admin] PATH_INFO: `/v3/projects/2e85a0ae928f421fb2fec1285c4c1941` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:39.816749 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c7b6a-7756-43b6-bc93-119a01d8fc83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e85a0ae928f421fb2fec1285c4c1941)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:39.818698 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c7b6a-7756-43b6-bc93-119a01d8fc83 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:39.884670 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:39.933004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-9e7c7b6a-7756-43b6-bc93-119a01d8fc83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e85a0ae928f421fb2fec1285c4c1941', 'request_id': 'req-9e7c7b6a-7756-43b6-bc93-119a01d8fc83'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:39.952677 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-9e7c7b6a-7756-43b6-bc93-119a01d8fc83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e85a0ae928f421fb2fec1285c4c1941', 'request_id': 'req-9e7c7b6a-7756-43b6-bc93-119a01d8fc83'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:40.031674 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-9e7c7b6a-7756-43b6-bc93-119a01d8fc83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e85a0ae928f421fb2fec1285c4c1941', 'request_id': 'req-9e7c7b6a-7756-43b6-bc93-119a01d8fc83'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:40.046688 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-9e7c7b6a-7756-43b6-bc93-119a01d8fc83 admin admin] The token cache is being invalidate because project 2e85a0ae928f421fb2fec1285c4c1941 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:14:40.047316 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-9e7c7b6a-7756-43b6-bc93-119a01d8fc83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:40.048003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-9e7c7b6a-7756-43b6-bc93-119a01d8fc83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:40.053369 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1123/2239] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:14:39 2022] DELETE /identity/v3/projects/2e85a0ae928f421fb2fec1285c4c1941 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:40.059688 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70741d79-646e-4560-9f34-624a1da2d220 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:40.061636 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-70741d79-646e-4560-9f34-624a1da2d220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:40.078464 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70741d79-646e-4560-9f34-624a1da2d220 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:40.136926 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c7b6a-7756-43b6-bc93-119a01d8fc83 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:40.137870 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-9e7c7b6a-7756-43b6-bc93-119a01d8fc83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:40.184025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fdc644-796c-43b6-825f-b5eb036bbeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:40.184707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdc644-796c-43b6-825f-b5eb036bbeb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:40.185052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdc644-796c-43b6-825f-b5eb036bbeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:40.185343 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdc644-796c-43b6-825f-b5eb036bbeb6 admin admin] PATH_INFO: `/v3/users/035cf1925dd84867a0e68fc392023b17` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:40.187273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fdc644-796c-43b6-825f-b5eb036bbeb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=035cf1925dd84867a0e68fc392023b17)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:40.192284 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fdc644-796c-43b6-825f-b5eb036bbeb6 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:40.271216 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c7b6a-7756-43b6-bc93-119a01d8fc83 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:40.271216 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62d668-126e-444c-91af-9089cfec58cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:40.275445 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62d668-126e-444c-91af-9089cfec58cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:40.275445 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62d668-126e-444c-91af-9089cfec58cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:40.275445 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62d668-126e-444c-91af-9089cfec58cc admin admin] PATH_INFO: `/v3/projects/9c362a83db8e4eacaf885de9779eba7a` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:40.276769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f62d668-126e-444c-91af-9089cfec58cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c362a83db8e4eacaf885de9779eba7a)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:40.279402 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f62d668-126e-444c-91af-9089cfec58cc admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:40.447399 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-29fdc644-796c-43b6-825f-b5eb036bbeb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '035cf1925dd84867a0e68fc392023b17'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:40.449282 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-0f62d668-126e-444c-91af-9089cfec58cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c362a83db8e4eacaf885de9779eba7a', 'request_id': 'req-0f62d668-126e-444c-91af-9089cfec58cc'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:40.458678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-0f62d668-126e-444c-91af-9089cfec58cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c362a83db8e4eacaf885de9779eba7a', 'request_id': 'req-0f62d668-126e-444c-91af-9089cfec58cc'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:40.504677 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-29fdc644-796c-43b6-825f-b5eb036bbeb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '035cf1925dd84867a0e68fc392023b17'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:40.505313 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-29fdc644-796c-43b6-825f-b5eb036bbeb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '035cf1925dd84867a0e68fc392023b17'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:40.519213 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-29fdc644-796c-43b6-825f-b5eb036bbeb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '035cf1925dd84867a0e68fc392023b17'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:40.519720 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-29fdc644-796c-43b6-825f-b5eb036bbeb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '035cf1925dd84867a0e68fc392023b17'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:40.533139 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1117/2240] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:14:40 2022] DELETE /identity/v3/users/035cf1925dd84867a0e68fc392023b17 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:40.550789 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-0f62d668-126e-444c-91af-9089cfec58cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c362a83db8e4eacaf885de9779eba7a', 'request_id': 'req-0f62d668-126e-444c-91af-9089cfec58cc'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:40.566746 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-0f62d668-126e-444c-91af-9089cfec58cc admin admin] The token cache is being invalidate because project 9c362a83db8e4eacaf885de9779eba7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:14:40.567264 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-0f62d668-126e-444c-91af-9089cfec58cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:40.568074 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-0f62d668-126e-444c-91af-9089cfec58cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:40.572163 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1124/2241] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:14:40 2022] DELETE /identity/v3/projects/9c362a83db8e4eacaf885de9779eba7a => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:40.578747 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed44c1-63be-4754-83d9-0f6afe6cab65 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:40.579078 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed44c1-63be-4754-83d9-0f6afe6cab65 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:40.579360 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed44c1-63be-4754-83d9-0f6afe6cab65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:40.602356 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-62ed44c1-63be-4754-83d9-0f6afe6cab65 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:40.603691 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-62ed44c1-63be-4754-83d9-0f6afe6cab65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:40.679034 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1118/2242] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:40.688706 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a67f4e-058d-4971-a550-323a0d755ad3 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:40.689726 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a67f4e-058d-4971-a550-323a0d755ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:40.689726 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a67f4e-058d-4971-a550-323a0d755ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:40.710570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-83a67f4e-058d-4971-a550-323a0d755ad3 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:40.712237 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-83a67f4e-058d-4971-a550-323a0d755ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:40.721374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1125/2243] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:40.725828 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed44c1-63be-4754-83d9-0f6afe6cab65 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:40.737706 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed44c1-63be-4754-83d9-0f6afe6cab65 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:40.742542 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba341766-097f-411e-8bb0-39bd679f5c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:40.743226 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba341766-097f-411e-8bb0-39bd679f5c84 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:40.743512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba341766-097f-411e-8bb0-39bd679f5c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:40.743791 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba341766-097f-411e-8bb0-39bd679f5c84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:40.744561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba341766-097f-411e-8bb0-39bd679f5c84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:14:40.745195 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba341766-097f-411e-8bb0-39bd679f5c84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:40.746465 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba341766-097f-411e-8bb0-39bd679f5c84 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:40.759827 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1119/2244] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:14:40.763863 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a67f4e-058d-4971-a550-323a0d755ad3 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:40.774310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a67f4e-058d-4971-a550-323a0d755ad3 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:40.779518 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79746405-460b-4849-9975-5a6246d7ef32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:40.780711 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79746405-460b-4849-9975-5a6246d7ef32 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:40.781068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79746405-460b-4849-9975-5a6246d7ef32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:40.781386 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79746405-460b-4849-9975-5a6246d7ef32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:40.782135 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79746405-460b-4849-9975-5a6246d7ef32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:40.784156 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79746405-460b-4849-9975-5a6246d7ef32 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:40.819091 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:40.931249 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1126/2245] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:14:40 2022] POST /identity/v3/projects => generated 367 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:40.932665 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba341766-097f-411e-8bb0-39bd679f5c84 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:40.943730 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba341766-097f-411e-8bb0-39bd679f5c84 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:41.004096 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e759d9f3-5f05-435d-b486-c5f576f556ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:41.005201 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759d9f3-5f05-435d-b486-c5f576f556ec admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:41.005343 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759d9f3-5f05-435d-b486-c5f576f556ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:41.005639 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759d9f3-5f05-435d-b486-c5f576f556ec admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:41.006309 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e759d9f3-5f05-435d-b486-c5f576f556ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:41.008282 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e759d9f3-5f05-435d-b486-c5f576f556ec admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:41.054690 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79746405-460b-4849-9975-5a6246d7ef32 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:41.056395 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-79746405-460b-4849-9975-5a6246d7ef32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:41.072183 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79746405-460b-4849-9975-5a6246d7ef32 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:41.078360 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4e66e-e7af-4176-8a1f-95c233409c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:41.082516 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4e66e-e7af-4176-8a1f-95c233409c19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:41.082516 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4e66e-e7af-4176-8a1f-95c233409c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:41.082516 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4e66e-e7af-4176-8a1f-95c233409c19 admin admin] PATH_INFO: `/v3/projects/e20e5ce9e0914a64a918dbcd9e541b11` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:41.082516 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c4e66e-e7af-4176-8a1f-95c233409c19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e20e5ce9e0914a64a918dbcd9e541b11)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:41.084207 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c4e66e-e7af-4176-8a1f-95c233409c19 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:41.145495 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1120/2246] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:14:40 2022] POST /identity/v3/users => generated 284 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:41.149978 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e759d9f3-5f05-435d-b486-c5f576f556ec admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:41.164688 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e759d9f3-5f05-435d-b486-c5f576f556ec admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:41.171981 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b003b-7478-4fb6-834c-250ccfab3801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:41.172929 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b003b-7478-4fb6-834c-250ccfab3801 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:41.173362 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b003b-7478-4fb6-834c-250ccfab3801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:41.173803 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b003b-7478-4fb6-834c-250ccfab3801 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:41.174684 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62b003b-7478-4fb6-834c-250ccfab3801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:41.176784 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62b003b-7478-4fb6-834c-250ccfab3801 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:41.188657 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:41.193814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1121/2247] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:14:41.197938 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b003b-7478-4fb6-834c-250ccfab3801 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:41.209737 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b003b-7478-4fb6-834c-250ccfab3801 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:41.214544 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cefc4c4-15f0-4e4b-9e47-b3b7a5ebf3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:41.215286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefc4c4-15f0-4e4b-9e47-b3b7a5ebf3d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:41.215573 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefc4c4-15f0-4e4b-9e47-b3b7a5ebf3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:41.215852 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefc4c4-15f0-4e4b-9e47-b3b7a5ebf3d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:41.216559 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cefc4c4-15f0-4e4b-9e47-b3b7a5ebf3d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:41.218328 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cefc4c4-15f0-4e4b-9e47-b3b7a5ebf3d3 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:41.230410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1122/2248] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:14:41.234483 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cefc4c4-15f0-4e4b-9e47-b3b7a5ebf3d3 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:41.245745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cefc4c4-15f0-4e4b-9e47-b3b7a5ebf3d3 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:41.252206 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed8a553-d665-4f1a-b24e-d493f7d5ce01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:41.252976 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8a553-d665-4f1a-b24e-d493f7d5ce01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:41.253264 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8a553-d665-4f1a-b24e-d493f7d5ce01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:41.253569 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8a553-d665-4f1a-b24e-d493f7d5ce01 admin admin] PATH_INFO: `/v3/projects/2c04e00c66d54983a8d7ba651faa7096/users/6d179f4195724b9e8e29fd5cb773c33a/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:41.270291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed8a553-d665-4f1a-b24e-d493f7d5ce01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c04e00c66d54983a8d7ba651faa7096, user_id=6d179f4195724b9e8e29fd5cb773c33a, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:41.278703 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed8a553-d665-4f1a-b24e-d493f7d5ce01 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:41.298680 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-82c4e66e-e7af-4176-8a1f-95c233409c19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e20e5ce9e0914a64a918dbcd9e541b11', 'request_id': 'req-82c4e66e-e7af-4176-8a1f-95c233409c19'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:41.306205 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-82c4e66e-e7af-4176-8a1f-95c233409c19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e20e5ce9e0914a64a918dbcd9e541b11', 'request_id': 'req-82c4e66e-e7af-4176-8a1f-95c233409c19'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:41.379814 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-82c4e66e-e7af-4176-8a1f-95c233409c19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e20e5ce9e0914a64a918dbcd9e541b11', 'request_id': 'req-82c4e66e-e7af-4176-8a1f-95c233409c19'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:41.388012 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1123/2249] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:14:41 2022] PUT /identity/v3/projects/2c04e00c66d54983a8d7ba651faa7096/users/6d179f4195724b9e8e29fd5cb773c33a/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:14:41.399717 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-82c4e66e-e7af-4176-8a1f-95c233409c19 admin admin] The token cache is being invalidate because project e20e5ce9e0914a64a918dbcd9e541b11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:14:41.399717 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-82c4e66e-e7af-4176-8a1f-95c233409c19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:41.399717 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-82c4e66e-e7af-4176-8a1f-95c233409c19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:41.400788 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1127/2250] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:14:41 2022] DELETE /identity/v3/projects/e20e5ce9e0914a64a918dbcd9e541b11 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:14:41.408148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f342535-4cdd-459d-bdf6-c87e63186efa None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:41.408712 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc4d06-c8a6-4a6e-b913-7152e85ff24f None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:41.408958 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f342535-4cdd-459d-bdf6-c87e63186efa None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:41.409193 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc4d06-c8a6-4a6e-b913-7152e85ff24f None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:41.409585 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f342535-4cdd-459d-bdf6-c87e63186efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:41.409702 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc4d06-c8a6-4a6e-b913-7152e85ff24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:41.439177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-4f342535-4cdd-459d-bdf6-c87e63186efa None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:41.442884 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-4f342535-4cdd-459d-bdf6-c87e63186efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:41.455309 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-1fcc4d06-c8a6-4a6e-b913-7152e85ff24f None None] MFA Rules not processed for user `6d179f4195724b9e8e29fd5cb773c33a`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:41.457497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1fcc4d06-c8a6-4a6e-b913-7152e85ff24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:41.513104 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1128/2251] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:41.521818 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b683586-ec5c-43fb-9ca0-d959a6ad9900 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:41.522380 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b683586-ec5c-43fb-9ca0-d959a6ad9900 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:41.522764 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b683586-ec5c-43fb-9ca0-d959a6ad9900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:41.545282 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1124/2252] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:41.546316 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-3b683586-ec5c-43fb-9ca0-d959a6ad9900 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:41.547744 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-3b683586-ec5c-43fb-9ca0-d959a6ad9900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:41.552728 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc4d06-c8a6-4a6e-b913-7152e85ff24f None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:41.554028 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1fcc4d06-c8a6-4a6e-b913-7152e85ff24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:41.555981 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1129/2253] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:41.560442 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b683586-ec5c-43fb-9ca0-d959a6ad9900 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:41.568336 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc4d06-c8a6-4a6e-b913-7152e85ff24f None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:41.569630 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b683586-ec5c-43fb-9ca0-d959a6ad9900 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:41.573423 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5972ca-7af8-408d-ab53-3a25d9820a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:41.574136 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5972ca-7af8-408d-ab53-3a25d9820a80 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:41.574490 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5972ca-7af8-408d-ab53-3a25d9820a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:41.574834 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5972ca-7af8-408d-ab53-3a25d9820a80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:41.575458 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319162c7-7f67-40fc-a63c-b286ccc321ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:41.575662 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5972ca-7af8-408d-ab53-3a25d9820a80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:41.576218 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319162c7-7f67-40fc-a63c-b286ccc321ef admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:41.576560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319162c7-7f67-40fc-a63c-b286ccc321ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:41.577518 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319162c7-7f67-40fc-a63c-b286ccc321ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:41.578306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319162c7-7f67-40fc-a63c-b286ccc321ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:14:41.578682 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319162c7-7f67-40fc-a63c-b286ccc321ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:41.579216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5972ca-7af8-408d-ab53-3a25d9820a80 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:41.580357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319162c7-7f67-40fc-a63c-b286ccc321ef admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:41.594654 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1130/2254] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 05:14:41.599315 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319162c7-7f67-40fc-a63c-b286ccc321ef admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:41.610516 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319162c7-7f67-40fc-a63c-b286ccc321ef admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:41.618047 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62bfe91-5b09-4266-8dcf-e7150f0238c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:41.619549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62bfe91-5b09-4266-8dcf-e7150f0238c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:41.619549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62bfe91-5b09-4266-8dcf-e7150f0238c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:41.619847 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62bfe91-5b09-4266-8dcf-e7150f0238c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:41.620560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62bfe91-5b09-4266-8dcf-e7150f0238c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:41.622713 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62bfe91-5b09-4266-8dcf-e7150f0238c2 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:41.664596 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1125/2255] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:14:41 2022] POST /identity/v3/projects => generated 367 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:14:41.665657 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5972ca-7af8-408d-ab53-3a25d9820a80 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:41.744697 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5972ca-7af8-408d-ab53-3a25d9820a80 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:41.749347 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c339e7f-8520-4446-9caa-d7db20dd80e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:41.750215 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c339e7f-8520-4446-9caa-d7db20dd80e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:41.750330 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c339e7f-8520-4446-9caa-d7db20dd80e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:41.750564 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c339e7f-8520-4446-9caa-d7db20dd80e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:41.751402 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c339e7f-8520-4446-9caa-d7db20dd80e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:41.753377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c339e7f-8520-4446-9caa-d7db20dd80e2 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:41.769341 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1131/2256] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:14:41 2022] POST /identity/v3/projects => generated 389 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:41.775966 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62bfe91-5b09-4266-8dcf-e7150f0238c2 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:41.845060 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62bfe91-5b09-4266-8dcf-e7150f0238c2 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:41.850224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb560e-b092-4038-bdd0-cffcd41f1ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:41.850868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb560e-b092-4038-bdd0-cffcd41f1ecd admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:41.851174 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb560e-b092-4038-bdd0-cffcd41f1ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:41.857744 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb560e-b092-4038-bdd0-cffcd41f1ecd admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:41.857744 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb560e-b092-4038-bdd0-cffcd41f1ecd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:41.857744 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb560e-b092-4038-bdd0-cffcd41f1ecd admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:41.857999 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1126/2257] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:14:41 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:41.862023 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c339e7f-8520-4446-9caa-d7db20dd80e2 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:41.872781 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c339e7f-8520-4446-9caa-d7db20dd80e2 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:41.878007 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab19537-6fd0-41a2-8c3b-d55f1bf02df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:41.878628 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab19537-6fd0-41a2-8c3b-d55f1bf02df4 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:41.884794 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab19537-6fd0-41a2-8c3b-d55f1bf02df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:41.884794 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab19537-6fd0-41a2-8c3b-d55f1bf02df4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:41.884794 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab19537-6fd0-41a2-8c3b-d55f1bf02df4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:41.884794 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab19537-6fd0-41a2-8c3b-d55f1bf02df4 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:41.893384 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1127/2258] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:14:41.897539 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab19537-6fd0-41a2-8c3b-d55f1bf02df4 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:41.908404 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab19537-6fd0-41a2-8c3b-d55f1bf02df4 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:41.917971 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52100961-7fc6-47e2-b0e3-f6b542d553a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:41.917971 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52100961-7fc6-47e2-b0e3-f6b542d553a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:41.917971 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52100961-7fc6-47e2-b0e3-f6b542d553a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:41.917971 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52100961-7fc6-47e2-b0e3-f6b542d553a0 admin admin] PATH_INFO: `/v3/projects/3966ac3c5ad14f1ea5dff1a735e46f6d/users/a6e75cd9663c4971b4280411ca5c446c/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:41.929920 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52100961-7fc6-47e2-b0e3-f6b542d553a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3966ac3c5ad14f1ea5dff1a735e46f6d, user_id=a6e75cd9663c4971b4280411ca5c446c, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:41.932561 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52100961-7fc6-47e2-b0e3-f6b542d553a0 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:42.004183 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1132/2259] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:14:41 2022] POST /identity/v3/users => generated 295 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:42.006880 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb560e-b092-4038-bdd0-cffcd41f1ecd admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:42.017938 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb560e-b092-4038-bdd0-cffcd41f1ecd admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:42.023866 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49163a44-c5df-4594-a3d7-2149297a183e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:42.024621 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49163a44-c5df-4594-a3d7-2149297a183e admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:42.024981 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49163a44-c5df-4594-a3d7-2149297a183e admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:42.025329 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49163a44-c5df-4594-a3d7-2149297a183e admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:42.026235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49163a44-c5df-4594-a3d7-2149297a183e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:42.028006 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49163a44-c5df-4594-a3d7-2149297a183e admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:42.039882 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1133/2260] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:14:42.043997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49163a44-c5df-4594-a3d7-2149297a183e admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:42.055649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49163a44-c5df-4594-a3d7-2149297a183e admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:42.058205 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1128/2261] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:14:41 2022] PUT /identity/v3/projects/3966ac3c5ad14f1ea5dff1a735e46f6d/users/a6e75cd9663c4971b4280411ca5c446c/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:14:42.062885 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52100961-7fc6-47e2-b0e3-f6b542d553a0 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:42.123194 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a197171-25be-4442-bb46-106cac158bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:42.124013 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a197171-25be-4442-bb46-106cac158bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:42.124298 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a197171-25be-4442-bb46-106cac158bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:42.124572 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a197171-25be-4442-bb46-106cac158bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:42.125294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a197171-25be-4442-bb46-106cac158bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:42.126963 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a197171-25be-4442-bb46-106cac158bd9 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:42.137015 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1134/2262] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:14:42.141927 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a197171-25be-4442-bb46-106cac158bd9 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:42.144271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52100961-7fc6-47e2-b0e3-f6b542d553a0 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:42.151123 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1932fd3-cdc6-4854-bb55-7ce53fa24371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:42.155735 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1932fd3-cdc6-4854-bb55-7ce53fa24371 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:42.155867 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a197171-25be-4442-bb46-106cac158bd9 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:42.155967 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1932fd3-cdc6-4854-bb55-7ce53fa24371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:42.155967 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1932fd3-cdc6-4854-bb55-7ce53fa24371 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:42.155967 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1932fd3-cdc6-4854-bb55-7ce53fa24371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:42.155967 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1932fd3-cdc6-4854-bb55-7ce53fa24371 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:42.157602 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060e5d5-e20f-426a-a50c-6b6754488a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:42.158282 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060e5d5-e20f-426a-a50c-6b6754488a23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:42.158566 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060e5d5-e20f-426a-a50c-6b6754488a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:42.158870 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060e5d5-e20f-426a-a50c-6b6754488a23 admin admin] PATH_INFO: `/v3/projects/973189f7b53a4b86a32d72eb6010a3fd/users/a4edc60562544cf6bd05847baf067deb/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:42.163186 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:42.170527 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1129/2263] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:14:42.171790 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1932fd3-cdc6-4854-bb55-7ce53fa24371 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:42.181605 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060e5d5-e20f-426a-a50c-6b6754488a23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=973189f7b53a4b86a32d72eb6010a3fd, user_id=a4edc60562544cf6bd05847baf067deb, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:42.182210 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1932fd3-cdc6-4854-bb55-7ce53fa24371 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:42.182826 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060e5d5-e20f-426a-a50c-6b6754488a23 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:42.187169 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc46d1-1b69-4202-bea4-da2adb089690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:42.187846 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc46d1-1b69-4202-bea4-da2adb089690 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:42.188187 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc46d1-1b69-4202-bea4-da2adb089690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:42.188412 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc46d1-1b69-4202-bea4-da2adb089690 admin admin] PATH_INFO: `/v3/projects/3966ac3c5ad14f1ea5dff1a735e46f6d/users/a6e75cd9663c4971b4280411ca5c446c/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:42.190385 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc46d1-1b69-4202-bea4-da2adb089690 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3966ac3c5ad14f1ea5dff1a735e46f6d, user_id=a6e75cd9663c4971b4280411ca5c446c, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:42.193090 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc46d1-1b69-4202-bea4-da2adb089690 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:42.203945 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1130/2264] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:14:42 2022] PUT /identity/v3/projects/3966ac3c5ad14f1ea5dff1a735e46f6d/users/a6e75cd9663c4971b4280411ca5c446c/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:42.214769 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e8ab5-8b28-493b-ba02-b01305a63a11 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:42.215093 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e8ab5-8b28-493b-ba02-b01305a63a11 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:42.215379 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e8ab5-8b28-493b-ba02-b01305a63a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:42.243934 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-da8e8ab5-8b28-493b-ba02-b01305a63a11 None None] MFA Rules not processed for user `a6e75cd9663c4971b4280411ca5c446c`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:42.245310 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-da8e8ab5-8b28-493b-ba02-b01305a63a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:42.306768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1135/2265] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:14:42 2022] PUT /identity/v3/projects/973189f7b53a4b86a32d72eb6010a3fd/users/a4edc60562544cf6bd05847baf067deb/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:42.317241 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1131/2266] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:42 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:14:42.331600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060e5d5-e20f-426a-a50c-6b6754488a23 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:42.333469 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-b060e5d5-e20f-426a-a50c-6b6754488a23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:42.378822 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3112b86-0108-489d-aab3-3b6a91c84ea0 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:42.379177 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3112b86-0108-489d-aab3-3b6a91c84ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:42.379466 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3112b86-0108-489d-aab3-3b6a91c84ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:42.403769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060e5d5-e20f-426a-a50c-6b6754488a23 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:42.410741 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-b3112b86-0108-489d-aab3-3b6a91c84ea0 None None] MFA Rules not processed for user `a4edc60562544cf6bd05847baf067deb`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:42.411575 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbdb8f7-eab1-46f0-bc17-23f80be062a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:42.412061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b3112b86-0108-489d-aab3-3b6a91c84ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:42.412217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbdb8f7-eab1-46f0-bc17-23f80be062a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:42.412693 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbdb8f7-eab1-46f0-bc17-23f80be062a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:42.412978 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbdb8f7-eab1-46f0-bc17-23f80be062a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:42.484691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbdb8f7-eab1-46f0-bc17-23f80be062a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:42.486393 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbdb8f7-eab1-46f0-bc17-23f80be062a5 service cinder] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:42.490903 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1132/2267] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:42 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:14:42.499995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3112b86-0108-489d-aab3-3b6a91c84ea0 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:42.502301 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b3112b86-0108-489d-aab3-3b6a91c84ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:42.511168 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1136/2268] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:14:42 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:14:42.515377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3112b86-0108-489d-aab3-3b6a91c84ea0 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:42.519113 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85dc94-24d2-4cc5-b38f-5263197574d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:42.519939 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85dc94-24d2-4cc5-b38f-5263197574d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:42.520198 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85dc94-24d2-4cc5-b38f-5263197574d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:42.520482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85dc94-24d2-4cc5-b38f-5263197574d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:42.529707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba85dc94-24d2-4cc5-b38f-5263197574d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:42.531947 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba85dc94-24d2-4cc5-b38f-5263197574d7 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:42.541365 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1133/2269] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:14:42 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:14:42.577572 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:42.725333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbdb8f7-eab1-46f0-bc17-23f80be062a5 service cinder] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:42.734063 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbdb8f7-eab1-46f0-bc17-23f80be062a5 service cinder] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:42.737816 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da1def-4985-4d45-ba9c-d5b13640221f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:42.738375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da1def-4985-4d45-ba9c-d5b13640221f service cinder] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:42.738663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da1def-4985-4d45-ba9c-d5b13640221f service cinder] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:42.738978 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da1def-4985-4d45-ba9c-d5b13640221f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:42.808172 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7da1def-4985-4d45-ba9c-d5b13640221f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:42.809749 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7da1def-4985-4d45-ba9c-d5b13640221f service cinder] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:42.819738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1137/2270] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:14:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:14:43.013391 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:43.162655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:45.247327 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85dc94-24d2-4cc5-b38f-5263197574d7 service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:45.249556 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ba85dc94-24d2-4cc5-b38f-5263197574d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:45.347105 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85dc94-24d2-4cc5-b38f-5263197574d7 service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:45.360336 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fdfc5-6edb-43f3-8c32-8e3a11e89d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:45.360991 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fdfc5-6edb-43f3-8c32-8e3a11e89d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:45.361272 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fdfc5-6edb-43f3-8c32-8e3a11e89d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:45.361627 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fdfc5-6edb-43f3-8c32-8e3a11e89d50 admin admin] PATH_INFO: `/v3/users/6203acf97abc4fb18f8220f8ed34afc1` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:45.363839 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703fdfc5-6edb-43f3-8c32-8e3a11e89d50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6203acf97abc4fb18f8220f8ed34afc1)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:45.365773 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703fdfc5-6edb-43f3-8c32-8e3a11e89d50 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:45.761668 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-703fdfc5-6edb-43f3-8c32-8e3a11e89d50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6203acf97abc4fb18f8220f8ed34afc1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:45.915586 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-703fdfc5-6edb-43f3-8c32-8e3a11e89d50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6203acf97abc4fb18f8220f8ed34afc1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:45.916249 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-703fdfc5-6edb-43f3-8c32-8e3a11e89d50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6203acf97abc4fb18f8220f8ed34afc1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:45.918602 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:45.929067 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-703fdfc5-6edb-43f3-8c32-8e3a11e89d50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6203acf97abc4fb18f8220f8ed34afc1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:45.929696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-703fdfc5-6edb-43f3-8c32-8e3a11e89d50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6203acf97abc4fb18f8220f8ed34afc1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:45.941872 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1134/2271] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:14:45 2022] DELETE /identity/v3/users/6203acf97abc4fb18f8220f8ed34afc1 => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:14:45.946656 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59183a3-fc8a-47e0-b8f9-30ff0ed4012f None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:45.947236 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59183a3-fc8a-47e0-b8f9-30ff0ed4012f None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:45.947354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59183a3-fc8a-47e0-b8f9-30ff0ed4012f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:45.967982 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-d59183a3-fc8a-47e0-b8f9-30ff0ed4012f None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:45.969261 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-d59183a3-fc8a-47e0-b8f9-30ff0ed4012f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:46.010792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:46.030458 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1138/2272] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:46.040263 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fdfc5-6edb-43f3-8c32-8e3a11e89d50 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:46.041671 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-703fdfc5-6edb-43f3-8c32-8e3a11e89d50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:46.108398 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fdfc5-6edb-43f3-8c32-8e3a11e89d50 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:46.112217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce390f3-1c49-4012-a044-76c55ba0d859 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:46.112698 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce390f3-1c49-4012-a044-76c55ba0d859 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:46.112977 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce390f3-1c49-4012-a044-76c55ba0d859 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:46.113249 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce390f3-1c49-4012-a044-76c55ba0d859 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:46.124391 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce390f3-1c49-4012-a044-76c55ba0d859 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:46.126266 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce390f3-1c49-4012-a044-76c55ba0d859 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:46.140433 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1135/2273] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:14:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:14:46.667595 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59183a3-fc8a-47e0-b8f9-30ff0ed4012f None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:46.669678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-d59183a3-fc8a-47e0-b8f9-30ff0ed4012f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:46.683517 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59183a3-fc8a-47e0-b8f9-30ff0ed4012f None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:46.688411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70bef7-dd90-4d60-ba1a-3dd3ed655808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:46.689195 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70bef7-dd90-4d60-ba1a-3dd3ed655808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:46.689529 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70bef7-dd90-4d60-ba1a-3dd3ed655808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:46.689755 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70bef7-dd90-4d60-ba1a-3dd3ed655808 admin admin] PATH_INFO: `/v3/projects/2a9a04efa8e144e48b8a671c51fdaa5e` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:46.691146 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e70bef7-dd90-4d60-ba1a-3dd3ed655808 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a9a04efa8e144e48b8a671c51fdaa5e)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:46.693059 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e70bef7-dd90-4d60-ba1a-3dd3ed655808 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:46.820385 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-3e70bef7-dd90-4d60-ba1a-3dd3ed655808 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a9a04efa8e144e48b8a671c51fdaa5e', 'request_id': 'req-3e70bef7-dd90-4d60-ba1a-3dd3ed655808'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:46.827009 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-3e70bef7-dd90-4d60-ba1a-3dd3ed655808 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a9a04efa8e144e48b8a671c51fdaa5e', 'request_id': 'req-3e70bef7-dd90-4d60-ba1a-3dd3ed655808'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:46.932272 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-3e70bef7-dd90-4d60-ba1a-3dd3ed655808 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a9a04efa8e144e48b8a671c51fdaa5e', 'request_id': 'req-3e70bef7-dd90-4d60-ba1a-3dd3ed655808'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:46.943582 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-3e70bef7-dd90-4d60-ba1a-3dd3ed655808 admin admin] The token cache is being invalidate because project 2a9a04efa8e144e48b8a671c51fdaa5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:14:46.943869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-3e70bef7-dd90-4d60-ba1a-3dd3ed655808 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:46.944524 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-3e70bef7-dd90-4d60-ba1a-3dd3ed655808 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:46.946036 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1139/2274] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:14:46 2022] DELETE /identity/v3/projects/2a9a04efa8e144e48b8a671c51fdaa5e => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:46.951120 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2fbe3-3f15-4230-81f5-0fc9770ebf37 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:46.951580 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2fbe3-3f15-4230-81f5-0fc9770ebf37 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:46.951723 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2fbe3-3f15-4230-81f5-0fc9770ebf37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:46.972368 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-c7e2fbe3-3f15-4230-81f5-0fc9770ebf37 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:46.973574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c7e2fbe3-3f15-4230-81f5-0fc9770ebf37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:47.033995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1136/2275] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:47.043529 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7fa096-f492-40bd-baa9-a114be08ac3b None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:47.044029 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7fa096-f492-40bd-baa9-a114be08ac3b None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:47.044121 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7fa096-f492-40bd-baa9-a114be08ac3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:47.065611 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-5d7fa096-f492-40bd-baa9-a114be08ac3b None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:47.067399 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-5d7fa096-f492-40bd-baa9-a114be08ac3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:47.076736 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1140/2276] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:47.081113 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e2fbe3-3f15-4230-81f5-0fc9770ebf37 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:47.091570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e2fbe3-3f15-4230-81f5-0fc9770ebf37 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:47.097106 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce01a8-caa5-4d7a-ba1c-45d79df27df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:47.097727 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce01a8-caa5-4d7a-ba1c-45d79df27df7 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:47.098031 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce01a8-caa5-4d7a-ba1c-45d79df27df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:47.098590 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce01a8-caa5-4d7a-ba1c-45d79df27df7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:47.099346 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce01a8-caa5-4d7a-ba1c-45d79df27df7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:14:47.099684 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce01a8-caa5-4d7a-ba1c-45d79df27df7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:47.101288 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce01a8-caa5-4d7a-ba1c-45d79df27df7 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:47.114098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1137/2277] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 05:14:47.118492 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7fa096-f492-40bd-baa9-a114be08ac3b None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:47.128003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7fa096-f492-40bd-baa9-a114be08ac3b None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:47.132770 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91a7d6-d3e2-4bb7-a81f-5f1931d36618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:47.136304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91a7d6-d3e2-4bb7-a81f-5f1931d36618 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:47.136304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91a7d6-d3e2-4bb7-a81f-5f1931d36618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:47.136304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91a7d6-d3e2-4bb7-a81f-5f1931d36618 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:47.136304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91a7d6-d3e2-4bb7-a81f-5f1931d36618 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:47.136694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91a7d6-d3e2-4bb7-a81f-5f1931d36618 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:47.226519 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1141/2278] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:14:47 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:47.230759 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce01a8-caa5-4d7a-ba1c-45d79df27df7 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:47.241582 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce01a8-caa5-4d7a-ba1c-45d79df27df7 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:47.318150 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb94bcc0-5be6-4544-b620-f319a0da3ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:47.318766 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94bcc0-5be6-4544-b620-f319a0da3ed1 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:47.319062 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94bcc0-5be6-4544-b620-f319a0da3ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:47.319341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94bcc0-5be6-4544-b620-f319a0da3ed1 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:47.320111 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb94bcc0-5be6-4544-b620-f319a0da3ed1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:47.322057 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb94bcc0-5be6-4544-b620-f319a0da3ed1 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:47.337126 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:47.462187 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1138/2279] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:14:47 2022] POST /identity/v3/users => generated 286 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:47.463836 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91a7d6-d3e2-4bb7-a81f-5f1931d36618 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:47.476622 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91a7d6-d3e2-4bb7-a81f-5f1931d36618 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:47.483770 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbe557-46a8-4c6d-9ee7-5ce9fa704d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:47.484401 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbe557-46a8-4c6d-9ee7-5ce9fa704d91 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:47.484686 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbe557-46a8-4c6d-9ee7-5ce9fa704d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:47.484962 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbe557-46a8-4c6d-9ee7-5ce9fa704d91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:47.485665 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dbe557-46a8-4c6d-9ee7-5ce9fa704d91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:47.487318 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dbe557-46a8-4c6d-9ee7-5ce9fa704d91 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:47.497861 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1142/2280] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:14:47.501974 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb94bcc0-5be6-4544-b620-f319a0da3ed1 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:47.512087 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb94bcc0-5be6-4544-b620-f319a0da3ed1 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:47.517693 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbffaab9-c485-45fe-ae21-13cf1e4d96dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:47.518311 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffaab9-c485-45fe-ae21-13cf1e4d96dd admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:47.518591 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffaab9-c485-45fe-ae21-13cf1e4d96dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:47.519039 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffaab9-c485-45fe-ae21-13cf1e4d96dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:47.519601 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbffaab9-c485-45fe-ae21-13cf1e4d96dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:47.521211 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbffaab9-c485-45fe-ae21-13cf1e4d96dd admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:47.534029 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1139/2281] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:14:47.538659 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbe557-46a8-4c6d-9ee7-5ce9fa704d91 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:47.549361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbe557-46a8-4c6d-9ee7-5ce9fa704d91 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:47.554492 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82894cd1-a49b-4d64-a007-38a6cd57f2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:47.555271 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82894cd1-a49b-4d64-a007-38a6cd57f2a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:47.555625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82894cd1-a49b-4d64-a007-38a6cd57f2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:47.556026 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82894cd1-a49b-4d64-a007-38a6cd57f2a5 admin admin] PATH_INFO: `/v3/projects/6ff6f2c611f44256a7d85a8d99981d9d/users/f145cffb3c4d4965a35553b95b316f56/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:47.582148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82894cd1-a49b-4d64-a007-38a6cd57f2a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ff6f2c611f44256a7d85a8d99981d9d, user_id=f145cffb3c4d4965a35553b95b316f56, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:47.585174 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82894cd1-a49b-4d64-a007-38a6cd57f2a5 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:47.653769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1143/2282] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:14:47 2022] PUT /identity/v3/projects/6ff6f2c611f44256a7d85a8d99981d9d/users/f145cffb3c4d4965a35553b95b316f56/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:47.662904 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39dac6-54d3-4e57-b178-d9ec4c08c632 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:47.662904 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39dac6-54d3-4e57-b178-d9ec4c08c632 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:47.662904 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39dac6-54d3-4e57-b178-d9ec4c08c632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:47.682543 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-ed39dac6-54d3-4e57-b178-d9ec4c08c632 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:47.683919 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ed39dac6-54d3-4e57-b178-d9ec4c08c632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:47.764313 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1140/2283] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:47.775563 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82894cd1-a49b-4d64-a007-38a6cd57f2a5 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:47.779955 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-82894cd1-a49b-4d64-a007-38a6cd57f2a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:47.834227 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82894cd1-a49b-4d64-a007-38a6cd57f2a5 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:47.838271 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4433117-640e-4e75-9352-c6d5a7f5884f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:47.838937 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4433117-640e-4e75-9352-c6d5a7f5884f service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:47.839222 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4433117-640e-4e75-9352-c6d5a7f5884f service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:47.839503 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4433117-640e-4e75-9352-c6d5a7f5884f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:47.850170 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4433117-640e-4e75-9352-c6d5a7f5884f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:47.851633 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4433117-640e-4e75-9352-c6d5a7f5884f service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:47.868407 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1144/2284] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:14:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:14:53.262551 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39dac6-54d3-4e57-b178-d9ec4c08c632 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:53.264572 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ed39dac6-54d3-4e57-b178-d9ec4c08c632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:53.291556 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39dac6-54d3-4e57-b178-d9ec4c08c632 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:53.297109 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad37ba-87b3-47ee-9a69-c6d66148c96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:53.298008 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad37ba-87b3-47ee-9a69-c6d66148c96c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:53.298295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad37ba-87b3-47ee-9a69-c6d66148c96c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:53.298579 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad37ba-87b3-47ee-9a69-c6d66148c96c admin admin] PATH_INFO: `/v3/users/6d179f4195724b9e8e29fd5cb773c33a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:53.299824 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaad37ba-87b3-47ee-9a69-c6d66148c96c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d179f4195724b9e8e29fd5cb773c33a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:53.301824 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaad37ba-87b3-47ee-9a69-c6d66148c96c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:53.326560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4433117-640e-4e75-9352-c6d5a7f5884f service neutron] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:53.328783 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-d4433117-640e-4e75-9352-c6d5a7f5884f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:53.341875 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4433117-640e-4e75-9352-c6d5a7f5884f service neutron] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:53.347556 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21d93a-410c-49b2-af0b-cf63a6f55bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:53.348277 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21d93a-410c-49b2-af0b-cf63a6f55bb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:53.348562 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21d93a-410c-49b2-af0b-cf63a6f55bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:53.348842 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21d93a-410c-49b2-af0b-cf63a6f55bb6 admin admin] PATH_INFO: `/v3/users/55dfb2ea79e44c4493dea3993ecd3018` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:53.350054 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca21d93a-410c-49b2-af0b-cf63a6f55bb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55dfb2ea79e44c4493dea3993ecd3018)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:53.352036 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca21d93a-410c-49b2-af0b-cf63a6f55bb6 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:53.353838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:53.511422 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-aaad37ba-87b3-47ee-9a69-c6d66148c96c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d179f4195724b9e8e29fd5cb773c33a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:53.612785 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-aaad37ba-87b3-47ee-9a69-c6d66148c96c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d179f4195724b9e8e29fd5cb773c33a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:53.616556 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-aaad37ba-87b3-47ee-9a69-c6d66148c96c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d179f4195724b9e8e29fd5cb773c33a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:53.634002 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-ca21d93a-410c-49b2-af0b-cf63a6f55bb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55dfb2ea79e44c4493dea3993ecd3018'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:53.634899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-aaad37ba-87b3-47ee-9a69-c6d66148c96c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d179f4195724b9e8e29fd5cb773c33a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:53.635915 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-aaad37ba-87b3-47ee-9a69-c6d66148c96c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d179f4195724b9e8e29fd5cb773c33a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:53.665374 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1141/2285] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:14:53 2022] DELETE /identity/v3/users/6d179f4195724b9e8e29fd5cb773c33a => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:53.671832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a26747-fce3-4797-8e97-19d048f744a0 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:53.672200 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a26747-fce3-4797-8e97-19d048f744a0 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:53.672587 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a26747-fce3-4797-8e97-19d048f744a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:53.693731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-d3a26747-fce3-4797-8e97-19d048f744a0 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:53.694865 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d3a26747-fce3-4797-8e97-19d048f744a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:53.753352 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:53.756103 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-ca21d93a-410c-49b2-af0b-cf63a6f55bb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55dfb2ea79e44c4493dea3993ecd3018'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:53.757401 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-ca21d93a-410c-49b2-af0b-cf63a6f55bb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55dfb2ea79e44c4493dea3993ecd3018'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:53.781450 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-ca21d93a-410c-49b2-af0b-cf63a6f55bb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55dfb2ea79e44c4493dea3993ecd3018'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:53.782193 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-ca21d93a-410c-49b2-af0b-cf63a6f55bb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55dfb2ea79e44c4493dea3993ecd3018'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:53.798686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1142/2286] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:14:53.801950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1145/2287] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:14:53 2022] DELETE /identity/v3/users/55dfb2ea79e44c4493dea3993ecd3018 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:14:53.806276 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92258985-67cd-47ac-b78c-13393b4eeb1e None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:53.806655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92258985-67cd-47ac-b78c-13393b4eeb1e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:53.806763 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92258985-67cd-47ac-b78c-13393b4eeb1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:53.832716 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21d93a-410c-49b2-af0b-cf63a6f55bb6 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:53.834410 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-ca21d93a-410c-49b2-af0b-cf63a6f55bb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:53.842028 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-92258985-67cd-47ac-b78c-13393b4eeb1e None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:53.850490 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-92258985-67cd-47ac-b78c-13393b4eeb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:53.935059 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1143/2288] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:53.938468 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21d93a-410c-49b2-af0b-cf63a6f55bb6 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:53.943404 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59545b2a-ea55-4d0e-8fcd-3b899e54c424 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:53.944112 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59545b2a-ea55-4d0e-8fcd-3b899e54c424 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:53.944421 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59545b2a-ea55-4d0e-8fcd-3b899e54c424 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:53.944695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59545b2a-ea55-4d0e-8fcd-3b899e54c424 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:53.946627 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-59545b2a-ea55-4d0e-8fcd-3b899e54c424 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:53.950184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92258985-67cd-47ac-b78c-13393b4eeb1e None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:53.952181 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-92258985-67cd-47ac-b78c-13393b4eeb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:53.975311 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59545b2a-ea55-4d0e-8fcd-3b899e54c424 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:53.975606 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92258985-67cd-47ac-b78c-13393b4eeb1e None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:53.975687 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59545b2a-ea55-4d0e-8fcd-3b899e54c424 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:53.980478 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144121e-8ffd-40ad-a26b-56f65b0147a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:53.980756 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144121e-8ffd-40ad-a26b-56f65b0147a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:53.981038 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144121e-8ffd-40ad-a26b-56f65b0147a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:53.981312 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144121e-8ffd-40ad-a26b-56f65b0147a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:53.985754 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1146/2289] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:14:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:14:53.993674 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7144121e-8ffd-40ad-a26b-56f65b0147a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:53.995220 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7144121e-8ffd-40ad-a26b-56f65b0147a9 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:54.007563 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1144/2290] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:14:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:14:54.721838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59545b2a-ea55-4d0e-8fcd-3b899e54c424 service neutron] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:54.723353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-59545b2a-ea55-4d0e-8fcd-3b899e54c424 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:54.748491 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59545b2a-ea55-4d0e-8fcd-3b899e54c424 service neutron] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:54.788802 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d183322-1977-4cb2-b5af-e7931c958a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:54.789477 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d183322-1977-4cb2-b5af-e7931c958a6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:54.789773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d183322-1977-4cb2-b5af-e7931c958a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:54.790054 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d183322-1977-4cb2-b5af-e7931c958a6c admin admin] PATH_INFO: `/v3/projects/d70415ca011a4138862bdcf9ba6e5b84` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:54.791360 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d183322-1977-4cb2-b5af-e7931c958a6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d70415ca011a4138862bdcf9ba6e5b84)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:54.794065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d183322-1977-4cb2-b5af-e7931c958a6c admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:54.936956 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-0d183322-1977-4cb2-b5af-e7931c958a6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd70415ca011a4138862bdcf9ba6e5b84', 'request_id': 'req-0d183322-1977-4cb2-b5af-e7931c958a6c'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:54.944834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-0d183322-1977-4cb2-b5af-e7931c958a6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd70415ca011a4138862bdcf9ba6e5b84', 'request_id': 'req-0d183322-1977-4cb2-b5af-e7931c958a6c'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:54.995972 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144121e-8ffd-40ad-a26b-56f65b0147a9 service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:54.998583 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7144121e-8ffd-40ad-a26b-56f65b0147a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:55.024420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-0d183322-1977-4cb2-b5af-e7931c958a6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd70415ca011a4138862bdcf9ba6e5b84', 'request_id': 'req-0d183322-1977-4cb2-b5af-e7931c958a6c'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:55.046437 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144121e-8ffd-40ad-a26b-56f65b0147a9 service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:55.046594 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-0d183322-1977-4cb2-b5af-e7931c958a6c admin admin] The token cache is being invalidate because project d70415ca011a4138862bdcf9ba6e5b84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:14:55.046594 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-0d183322-1977-4cb2-b5af-e7931c958a6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:55.046594 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-0d183322-1977-4cb2-b5af-e7931c958a6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:55.047127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1147/2291] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:14:54 2022] DELETE /identity/v3/projects/d70415ca011a4138862bdcf9ba6e5b84 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:55.052595 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6682fa5-25ab-4aee-a34f-063df76d5fab None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:55.052888 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6682fa5-25ab-4aee-a34f-063df76d5fab None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:55.053165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6682fa5-25ab-4aee-a34f-063df76d5fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:55.072657 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:55.092632 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-a6682fa5-25ab-4aee-a34f-063df76d5fab None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:55.094484 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-a6682fa5-25ab-4aee-a34f-063df76d5fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:55.147980 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c66dc-d8a9-4781-91d7-dac549adbb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:55.148798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c66dc-d8a9-4781-91d7-dac549adbb39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:55.149184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c66dc-d8a9-4781-91d7-dac549adbb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:55.149568 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c66dc-d8a9-4781-91d7-dac549adbb39 admin admin] PATH_INFO: `/v3/projects/2c04e00c66d54983a8d7ba651faa7096` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:55.151465 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c66dc-d8a9-4781-91d7-dac549adbb39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c04e00c66d54983a8d7ba651faa7096)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:55.155465 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c66dc-d8a9-4781-91d7-dac549adbb39 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:55.232088 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1148/2292] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:14:55.233748 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783d162-3f2d-4898-a391-098adeda7208 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:55.234821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783d162-3f2d-4898-a391-098adeda7208 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:55.234821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783d162-3f2d-4898-a391-098adeda7208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:55.266403 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-4783d162-3f2d-4898-a391-098adeda7208 None None] MFA Rules not processed for user `f145cffb3c4d4965a35553b95b316f56`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:55.267929 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-4783d162-3f2d-4898-a391-098adeda7208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:55.341991 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ce3c66dc-d8a9-4781-91d7-dac549adbb39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c04e00c66d54983a8d7ba651faa7096', 'request_id': 'req-ce3c66dc-d8a9-4781-91d7-dac549adbb39'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:55.366882 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1149/2293] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:55.366882 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114a785-92ed-4089-9c03-bbd6755c26ad None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:55.366882 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114a785-92ed-4089-9c03-bbd6755c26ad None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:55.366882 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114a785-92ed-4089-9c03-bbd6755c26ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:55.388900 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-8114a785-92ed-4089-9c03-bbd6755c26ad None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:55.388900 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-8114a785-92ed-4089-9c03-bbd6755c26ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:55.407289 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ce3c66dc-d8a9-4781-91d7-dac549adbb39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c04e00c66d54983a8d7ba651faa7096', 'request_id': 'req-ce3c66dc-d8a9-4781-91d7-dac549adbb39'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:55.414567 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ce3c66dc-d8a9-4781-91d7-dac549adbb39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c04e00c66d54983a8d7ba651faa7096', 'request_id': 'req-ce3c66dc-d8a9-4781-91d7-dac549adbb39'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:55.429066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ce3c66dc-d8a9-4781-91d7-dac549adbb39 admin admin] The token cache is being invalidate because project 2c04e00c66d54983a8d7ba651faa7096 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:14:55.429362 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ce3c66dc-d8a9-4781-91d7-dac549adbb39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:55.430041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ce3c66dc-d8a9-4781-91d7-dac549adbb39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:55.431646 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1145/2294] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:14:54 2022] DELETE /identity/v3/projects/2c04e00c66d54983a8d7ba651faa7096 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:55.434693 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c66dc-d8a9-4781-91d7-dac549adbb39 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:55.434693 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ce3c66dc-d8a9-4781-91d7-dac549adbb39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:55.448775 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:55.461778 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1150/2295] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:55.465996 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8114a785-92ed-4089-9c03-bbd6755c26ad None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:55.466930 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-8114a785-92ed-4089-9c03-bbd6755c26ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:55.479454 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8114a785-92ed-4089-9c03-bbd6755c26ad None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:55.486031 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f47a18-6afc-4aed-8c86-6db74bfe7454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:55.486745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f47a18-6afc-4aed-8c86-6db74bfe7454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:55.492209 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f47a18-6afc-4aed-8c86-6db74bfe7454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:55.492209 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f47a18-6afc-4aed-8c86-6db74bfe7454 admin admin] PATH_INFO: `/v3/users/a6e75cd9663c4971b4280411ca5c446c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:55.492209 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f47a18-6afc-4aed-8c86-6db74bfe7454 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6e75cd9663c4971b4280411ca5c446c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:55.492209 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f47a18-6afc-4aed-8c86-6db74bfe7454 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:55.508149 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c66dc-d8a9-4781-91d7-dac549adbb39 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:55.515746 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027222c-dfce-4f39-bac4-4dc221c083b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:55.515746 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027222c-dfce-4f39-bac4-4dc221c083b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:55.515746 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027222c-dfce-4f39-bac4-4dc221c083b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:55.515746 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027222c-dfce-4f39-bac4-4dc221c083b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:55.516192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e027222c-dfce-4f39-bac4-4dc221c083b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:55.517366 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e027222c-dfce-4f39-bac4-4dc221c083b7 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:55.623539 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1146/2296] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:14:55 2022] POST /identity/v3/projects => generated 371 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:55.625123 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027222c-dfce-4f39-bac4-4dc221c083b7 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:55.688636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-72f47a18-6afc-4aed-8c86-6db74bfe7454 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6e75cd9663c4971b4280411ca5c446c'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:55.701959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027222c-dfce-4f39-bac4-4dc221c083b7 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:55.707947 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a47b3be-09a0-4e45-a37a-d20b3aa61ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:55.708746 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47b3be-09a0-4e45-a37a-d20b3aa61ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:55.709034 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47b3be-09a0-4e45-a37a-d20b3aa61ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:55.709349 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47b3be-09a0-4e45-a37a-d20b3aa61ec0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:55.710053 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a47b3be-09a0-4e45-a37a-d20b3aa61ec0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:14:55.710403 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a47b3be-09a0-4e45-a37a-d20b3aa61ec0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:55.711990 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a47b3be-09a0-4e45-a37a-d20b3aa61ec0 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:55.724875 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1147/2297] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:14:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:14:55.725527 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a47b3be-09a0-4e45-a37a-d20b3aa61ec0 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:55.735376 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a47b3be-09a0-4e45-a37a-d20b3aa61ec0 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:55.740176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b4a63f-98e7-4e3e-90e2-f984661109a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:55.740785 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4a63f-98e7-4e3e-90e2-f984661109a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:55.741071 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4a63f-98e7-4e3e-90e2-f984661109a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:55.741354 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4a63f-98e7-4e3e-90e2-f984661109a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:55.742018 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b4a63f-98e7-4e3e-90e2-f984661109a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:55.743971 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b4a63f-98e7-4e3e-90e2-f984661109a4 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:55.766811 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-72f47a18-6afc-4aed-8c86-6db74bfe7454 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6e75cd9663c4971b4280411ca5c446c'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:55.767791 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-72f47a18-6afc-4aed-8c86-6db74bfe7454 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6e75cd9663c4971b4280411ca5c446c'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:55.781318 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-72f47a18-6afc-4aed-8c86-6db74bfe7454 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6e75cd9663c4971b4280411ca5c446c'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:55.782192 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-72f47a18-6afc-4aed-8c86-6db74bfe7454 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6e75cd9663c4971b4280411ca5c446c'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:55.789765 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:55.800743 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1151/2298] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:14:55 2022] DELETE /identity/v3/users/a6e75cd9663c4971b4280411ca5c446c => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:55.800743 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f47a18-6afc-4aed-8c86-6db74bfe7454 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:55.800743 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-72f47a18-6afc-4aed-8c86-6db74bfe7454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:55.872550 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1148/2299] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:14:55 2022] POST /identity/v3/users => generated 292 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:55.878802 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b4a63f-98e7-4e3e-90e2-f984661109a4 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:55.880647 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-08b4a63f-98e7-4e3e-90e2-f984661109a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:55.883954 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f47a18-6afc-4aed-8c86-6db74bfe7454 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:55.889323 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c6f44-f84c-436d-b8af-e5e1625fa10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:55.890557 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c6f44-f84c-436d-b8af-e5e1625fa10c admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:55.890880 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c6f44-f84c-436d-b8af-e5e1625fa10c admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:55.891194 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c6f44-f84c-436d-b8af-e5e1625fa10c admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:55.892012 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8c6f44-f84c-436d-b8af-e5e1625fa10c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:55.894246 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8c6f44-f84c-436d-b8af-e5e1625fa10c admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:55.908524 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b4a63f-98e7-4e3e-90e2-f984661109a4 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:55.913529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7a3fa2-74b2-4cca-b7cf-1d5f93f1ede4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:55.914324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a3fa2-74b2-4cca-b7cf-1d5f93f1ede4 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:55.914679 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a3fa2-74b2-4cca-b7cf-1d5f93f1ede4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:55.915088 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a3fa2-74b2-4cca-b7cf-1d5f93f1ede4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:55.915875 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7a3fa2-74b2-4cca-b7cf-1d5f93f1ede4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:55.918160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7a3fa2-74b2-4cca-b7cf-1d5f93f1ede4 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:55.933022 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1149/2300] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:55 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:14:55.933713 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7a3fa2-74b2-4cca-b7cf-1d5f93f1ede4 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:55.944770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7a3fa2-74b2-4cca-b7cf-1d5f93f1ede4 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:55.950370 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d116ff3-1df7-4fc8-bb19-f3f6b6a494d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:55.953999 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d116ff3-1df7-4fc8-bb19-f3f6b6a494d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:55.953999 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d116ff3-1df7-4fc8-bb19-f3f6b6a494d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:55.953999 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d116ff3-1df7-4fc8-bb19-f3f6b6a494d8 admin admin] PATH_INFO: `/v3/projects/2f9038a964ca4cd595ed4851dd9e6407/users/7a806f98beb5459085fd4cca1f28da96/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:55.968240 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d116ff3-1df7-4fc8-bb19-f3f6b6a494d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f9038a964ca4cd595ed4851dd9e6407, user_id=7a806f98beb5459085fd4cca1f28da96, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:55.971489 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d116ff3-1df7-4fc8-bb19-f3f6b6a494d8 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:56.019211 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1152/2301] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:14:55 2022] POST /identity/v3/projects => generated 385 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:56.020846 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c6f44-f84c-436d-b8af-e5e1625fa10c admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:56.093597 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1150/2302] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:14:55 2022] PUT /identity/v3/projects/2f9038a964ca4cd595ed4851dd9e6407/users/7a806f98beb5459085fd4cca1f28da96/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:14:56.101503 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d116ff3-1df7-4fc8-bb19-f3f6b6a494d8 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:56.126395 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c6f44-f84c-436d-b8af-e5e1625fa10c admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:56.162571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:56.182737 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:56.191730 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d116ff3-1df7-4fc8-bb19-f3f6b6a494d8 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:56.197331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06505876-bef2-49c0-b3d2-4d16eb15cfcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:56.198417 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06505876-bef2-49c0-b3d2-4d16eb15cfcb admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:56.198707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06505876-bef2-49c0-b3d2-4d16eb15cfcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:56.199063 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06505876-bef2-49c0-b3d2-4d16eb15cfcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:56.199722 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06505876-bef2-49c0-b3d2-4d16eb15cfcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:56.202309 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06505876-bef2-49c0-b3d2-4d16eb15cfcb admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:56.204828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a9497-0ddd-403d-8158-dd3aaaf96218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:56.205460 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a9497-0ddd-403d-8158-dd3aaaf96218 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:56.211302 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a9497-0ddd-403d-8158-dd3aaaf96218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:56.211302 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a9497-0ddd-403d-8158-dd3aaaf96218 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:56.211302 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a9497-0ddd-403d-8158-dd3aaaf96218 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:56.211302 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a9497-0ddd-403d-8158-dd3aaaf96218 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:56.218588 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1151/2303] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:14:56.223406 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06505876-bef2-49c0-b3d2-4d16eb15cfcb admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:56.234419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06505876-bef2-49c0-b3d2-4d16eb15cfcb admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:56.239740 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ee253-a41d-41a9-afe7-3957f482cacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:56.240392 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ee253-a41d-41a9-afe7-3957f482cacb admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:56.240675 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ee253-a41d-41a9-afe7-3957f482cacb admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:56.240955 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ee253-a41d-41a9-afe7-3957f482cacb admin admin] PATH_INFO: `/v3/projects/2f9038a964ca4cd595ed4851dd9e6407/users/7a806f98beb5459085fd4cca1f28da96/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:56.242983 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ee253-a41d-41a9-afe7-3957f482cacb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f9038a964ca4cd595ed4851dd9e6407, user_id=7a806f98beb5459085fd4cca1f28da96, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:56.245653 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ee253-a41d-41a9-afe7-3957f482cacb admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:56.261484 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1152/2304] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:14:56 2022] PUT /identity/v3/projects/2f9038a964ca4cd595ed4851dd9e6407/users/7a806f98beb5459085fd4cca1f28da96/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:56.340331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ee253-a41d-41a9-afe7-3957f482cacb admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:56.342407 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1d7ee253-a41d-41a9-afe7-3957f482cacb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:56.391207 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1153/2305] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:14:56 2022] POST /identity/v3/users => generated 293 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:56.395286 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a9497-0ddd-403d-8158-dd3aaaf96218 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:56.409733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a9497-0ddd-403d-8158-dd3aaaf96218 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:56.491334 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ee253-a41d-41a9-afe7-3957f482cacb admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:56.499128 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4931cf-a24d-4a07-b89f-52088a089f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:56.499796 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4931cf-a24d-4a07-b89f-52088a089f4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:56.500079 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4931cf-a24d-4a07-b89f-52088a089f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:56.500357 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4931cf-a24d-4a07-b89f-52088a089f4b admin admin] PATH_INFO: `/v3/projects/3966ac3c5ad14f1ea5dff1a735e46f6d` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:56.501821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4931cf-a24d-4a07-b89f-52088a089f4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3966ac3c5ad14f1ea5dff1a735e46f6d)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:56.503812 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4931cf-a24d-4a07-b89f-52088a089f4b admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:56.555087 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e8be7-22e0-46fb-b54a-4f9eaeb54ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:56.555859 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e8be7-22e0-46fb-b54a-4f9eaeb54ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:56.556195 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e8be7-22e0-46fb-b54a-4f9eaeb54ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:56.556778 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e8be7-22e0-46fb-b54a-4f9eaeb54ac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:56.557490 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81e8be7-22e0-46fb-b54a-4f9eaeb54ac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:56.559308 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81e8be7-22e0-46fb-b54a-4f9eaeb54ac6 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:56.581144 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1154/2306] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:14:56.584421 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e8be7-22e0-46fb-b54a-4f9eaeb54ac6 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:56.609425 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e8be7-22e0-46fb-b54a-4f9eaeb54ac6 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:56.654088 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-cc4931cf-a24d-4a07-b89f-52088a089f4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3966ac3c5ad14f1ea5dff1a735e46f6d', 'request_id': 'req-cc4931cf-a24d-4a07-b89f-52088a089f4b'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:56.723654 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-cc4931cf-a24d-4a07-b89f-52088a089f4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3966ac3c5ad14f1ea5dff1a735e46f6d', 'request_id': 'req-cc4931cf-a24d-4a07-b89f-52088a089f4b'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:56.732513 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-cc4931cf-a24d-4a07-b89f-52088a089f4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3966ac3c5ad14f1ea5dff1a735e46f6d', 'request_id': 'req-cc4931cf-a24d-4a07-b89f-52088a089f4b'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:56.765276 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-cc4931cf-a24d-4a07-b89f-52088a089f4b admin admin] The token cache is being invalidate because project 3966ac3c5ad14f1ea5dff1a735e46f6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:14:56.766422 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-cc4931cf-a24d-4a07-b89f-52088a089f4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:56.767329 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838bd66-196c-429d-ab17-ff76c2cfd7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:56.767988 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838bd66-196c-429d-ab17-ff76c2cfd7ce admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:56.770075 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838bd66-196c-429d-ab17-ff76c2cfd7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:56.770075 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838bd66-196c-429d-ab17-ff76c2cfd7ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:56.770075 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9838bd66-196c-429d-ab17-ff76c2cfd7ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:56.771277 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9838bd66-196c-429d-ab17-ff76c2cfd7ce admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:56.773036 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-cc4931cf-a24d-4a07-b89f-52088a089f4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:14:56.775596 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1153/2307] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:14:56 2022] DELETE /identity/v3/projects/3966ac3c5ad14f1ea5dff1a735e46f6d => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:56.779985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5bd31-db57-4390-b93c-a4db1cd68685 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:56.780720 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5bd31-db57-4390-b93c-a4db1cd68685 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:56.781004 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5bd31-db57-4390-b93c-a4db1cd68685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:56.795117 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1155/2308] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:14:56.806941 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838bd66-196c-429d-ab17-ff76c2cfd7ce admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:56.808365 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-73c5bd31-db57-4390-b93c-a4db1cd68685 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:56.809775 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-73c5bd31-db57-4390-b93c-a4db1cd68685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:56.809888 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-9838bd66-196c-429d-ab17-ff76c2cfd7ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:56.820988 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1154/2309] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:14:56.828837 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838bd66-196c-429d-ab17-ff76c2cfd7ce admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:56.831155 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8a303-2b31-4da7-a37c-e4d2d88da1b8 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:56.831155 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8a303-2b31-4da7-a37c-e4d2d88da1b8 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:56.831155 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8a303-2b31-4da7-a37c-e4d2d88da1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:56.834823 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed4a01-c052-4d6f-8ff4-3ce0a68d7ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:56.836383 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed4a01-c052-4d6f-8ff4-3ce0a68d7ebf admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:56.836691 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed4a01-c052-4d6f-8ff4-3ce0a68d7ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:56.836977 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed4a01-c052-4d6f-8ff4-3ce0a68d7ebf admin admin] PATH_INFO: `/v3/projects/7d7bf6baa2224b0d910133c5542608e1/users/79539af2999d48baa11734ced32dccb2/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:56.856145 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-c9d8a303-2b31-4da7-a37c-e4d2d88da1b8 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:56.857721 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c9d8a303-2b31-4da7-a37c-e4d2d88da1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:56.859892 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eed4a01-c052-4d6f-8ff4-3ce0a68d7ebf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d7bf6baa2224b0d910133c5542608e1, user_id=79539af2999d48baa11734ced32dccb2, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:56.866002 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eed4a01-c052-4d6f-8ff4-3ce0a68d7ebf admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:56.875895 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1155/2310] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:56.880496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8a303-2b31-4da7-a37c-e4d2d88da1b8 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:56.896480 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8a303-2b31-4da7-a37c-e4d2d88da1b8 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:56.904768 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb5430-57d7-465f-b3cb-258a5ffac6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:56.907154 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb5430-57d7-465f-b3cb-258a5ffac6d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:56.907862 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb5430-57d7-465f-b3cb-258a5ffac6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:56.908375 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb5430-57d7-465f-b3cb-258a5ffac6d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:56.909346 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcb5430-57d7-465f-b3cb-258a5ffac6d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:14:56.909856 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcb5430-57d7-465f-b3cb-258a5ffac6d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:56.912617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcb5430-57d7-465f-b3cb-258a5ffac6d0 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:56.940100 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1156/2311] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:14:56.941622 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb5430-57d7-465f-b3cb-258a5ffac6d0 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:56.985610 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb5430-57d7-465f-b3cb-258a5ffac6d0 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:56.985890 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1156/2312] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:14:56 2022] PUT /identity/v3/projects/7d7bf6baa2224b0d910133c5542608e1/users/79539af2999d48baa11734ced32dccb2/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:56.985890 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edf78a-c303-4def-802a-d88f5dcc57b6 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:56.985890 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edf78a-c303-4def-802a-d88f5dcc57b6 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:56.985890 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edf78a-c303-4def-802a-d88f5dcc57b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:57.025313 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:57.027151 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-35edf78a-c303-4def-802a-d88f5dcc57b6 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:57.028446 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-35edf78a-c303-4def-802a-d88f5dcc57b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:57.082111 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38c8d8-920e-4f94-bc14-8685e830545b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:57.083655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38c8d8-920e-4f94-bc14-8685e830545b admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:57.084264 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38c8d8-920e-4f94-bc14-8685e830545b admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:57.084814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38c8d8-920e-4f94-bc14-8685e830545b admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:57.085816 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc38c8d8-920e-4f94-bc14-8685e830545b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:57.088142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc38c8d8-920e-4f94-bc14-8685e830545b admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:57.091681 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:57.139256 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1157/2313] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:57.149461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edf78a-c303-4def-802a-d88f5dcc57b6 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:57.151237 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-35edf78a-c303-4def-802a-d88f5dcc57b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:57.230323 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edf78a-c303-4def-802a-d88f5dcc57b6 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:57.234363 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6944359-9211-492d-bc69-c7ba7e2f93ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:57.235031 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6944359-9211-492d-bc69-c7ba7e2f93ec service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:57.235358 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6944359-9211-492d-bc69-c7ba7e2f93ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:57.235636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6944359-9211-492d-bc69-c7ba7e2f93ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:57.245272 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6944359-9211-492d-bc69-c7ba7e2f93ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:57.246759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6944359-9211-492d-bc69-c7ba7e2f93ec service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:57.260285 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1157/2314] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:14:56 2022] POST /identity/v3/projects => generated 381 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:57.265771 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1158/2315] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:14:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:14:57.266591 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38c8d8-920e-4f94-bc14-8685e830545b admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:57.332817 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38c8d8-920e-4f94-bc14-8685e830545b admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:57.337523 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014091b8-9043-4c71-81a7-598ef1e2cee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:57.338209 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014091b8-9043-4c71-81a7-598ef1e2cee3 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:57.338493 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014091b8-9043-4c71-81a7-598ef1e2cee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:57.338771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014091b8-9043-4c71-81a7-598ef1e2cee3 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:57.339471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014091b8-9043-4c71-81a7-598ef1e2cee3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:57.341574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014091b8-9043-4c71-81a7-598ef1e2cee3 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:57.426508 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1158/2316] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:14:57 2022] POST /identity/v3/users => generated 291 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:57.431136 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6944359-9211-492d-bc69-c7ba7e2f93ec service neutron] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:57.444301 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6944359-9211-492d-bc69-c7ba7e2f93ec service neutron] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:57.458156 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ae195-cc6a-4e52-93f1-33f8d948c04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:57.463327 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ae195-cc6a-4e52-93f1-33f8d948c04e admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:57.463327 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ae195-cc6a-4e52-93f1-33f8d948c04e admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:57.463327 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ae195-cc6a-4e52-93f1-33f8d948c04e admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:57.463327 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ae195-cc6a-4e52-93f1-33f8d948c04e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:57.463327 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ae195-cc6a-4e52-93f1-33f8d948c04e admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:57.478631 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1159/2317] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:14:57.482656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014091b8-9043-4c71-81a7-598ef1e2cee3 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:57.496507 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014091b8-9043-4c71-81a7-598ef1e2cee3 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:57.505852 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316cf6f-831b-4e0d-9b6c-3ca69cd053d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:57.506936 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316cf6f-831b-4e0d-9b6c-3ca69cd053d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:57.507860 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316cf6f-831b-4e0d-9b6c-3ca69cd053d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:57.508358 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316cf6f-831b-4e0d-9b6c-3ca69cd053d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:57.509363 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d316cf6f-831b-4e0d-9b6c-3ca69cd053d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:57.512362 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d316cf6f-831b-4e0d-9b6c-3ca69cd053d3 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:57.529237 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1159/2318] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:14:57.533774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ae195-cc6a-4e52-93f1-33f8d948c04e admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:57.545194 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ae195-cc6a-4e52-93f1-33f8d948c04e admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:57.550020 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7841c0a5-ffd8-4f1c-a0eb-991d80ea682d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:57.550721 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841c0a5-ffd8-4f1c-a0eb-991d80ea682d admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:57.551046 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841c0a5-ffd8-4f1c-a0eb-991d80ea682d admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:57.551515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841c0a5-ffd8-4f1c-a0eb-991d80ea682d admin admin] PATH_INFO: `/v3/projects/65c39ce41fb6438bb8abfc09000141c0/users/407bc0300eac4db495e8bba5d6178a5e/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:57.568105 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7841c0a5-ffd8-4f1c-a0eb-991d80ea682d admin admin] RBAC: Authorizing `identity:create_grant(project_id=65c39ce41fb6438bb8abfc09000141c0, user_id=407bc0300eac4db495e8bba5d6178a5e, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:57.571101 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7841c0a5-ffd8-4f1c-a0eb-991d80ea682d admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:57.685902 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1160/2319] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:14:57 2022] PUT /identity/v3/projects/65c39ce41fb6438bb8abfc09000141c0/users/407bc0300eac4db495e8bba5d6178a5e/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:14:57.697906 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0b09a-dbf8-4f46-97ed-829842eae4a2 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:57.700146 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0b09a-dbf8-4f46-97ed-829842eae4a2 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:57.700743 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0b09a-dbf8-4f46-97ed-829842eae4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:57.745622 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-1bf0b09a-dbf8-4f46-97ed-829842eae4a2 None None] MFA Rules not processed for user `407bc0300eac4db495e8bba5d6178a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:57.753538 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1bf0b09a-dbf8-4f46-97ed-829842eae4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:57.916147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1160/2320] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:14:57.921408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7841c0a5-ffd8-4f1c-a0eb-991d80ea682d admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:58.009279 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7841c0a5-ffd8-4f1c-a0eb-991d80ea682d admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:58.015021 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccdc268-29fc-4cc0-977a-16f8d368678a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:58.015633 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccdc268-29fc-4cc0-977a-16f8d368678a admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:58.015916 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccdc268-29fc-4cc0-977a-16f8d368678a admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:58.016385 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccdc268-29fc-4cc0-977a-16f8d368678a admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:58.016890 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccdc268-29fc-4cc0-977a-16f8d368678a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:58.019021 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccdc268-29fc-4cc0-977a-16f8d368678a admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:58.132825 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1161/2321] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:14:57 2022] POST /identity/v3/projects => generated 381 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:58.137038 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf0b09a-dbf8-4f46-97ed-829842eae4a2 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:58.152832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf0b09a-dbf8-4f46-97ed-829842eae4a2 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:58.189883 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:58.251165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b95db-3efe-45e3-976c-af775977025a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:58.252312 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b95db-3efe-45e3-976c-af775977025a admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:58.252930 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b95db-3efe-45e3-976c-af775977025a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:58.253403 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b95db-3efe-45e3-976c-af775977025a admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:58.254313 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7b95db-3efe-45e3-976c-af775977025a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:58.257042 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7b95db-3efe-45e3-976c-af775977025a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:58.387744 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1161/2322] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:14:58 2022] POST /identity/v3/users => generated 297 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:14:58.390277 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccdc268-29fc-4cc0-977a-16f8d368678a admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:58.405461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccdc268-29fc-4cc0-977a-16f8d368678a admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:58.432351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d26bb5-1c5c-452b-bf8c-b5aafa22861d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:58.433909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d26bb5-1c5c-452b-bf8c-b5aafa22861d admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:58.434205 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d26bb5-1c5c-452b-bf8c-b5aafa22861d admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:58.434780 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d26bb5-1c5c-452b-bf8c-b5aafa22861d admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:58.439635 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d26bb5-1c5c-452b-bf8c-b5aafa22861d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:58.446419 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d26bb5-1c5c-452b-bf8c-b5aafa22861d admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:58.487656 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1162/2323] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:14:58.496700 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b95db-3efe-45e3-976c-af775977025a admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:58.518721 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b95db-3efe-45e3-976c-af775977025a admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:58.542089 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685eee1-cd81-49f0-8abb-0c5d0164a05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:58.542792 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685eee1-cd81-49f0-8abb-0c5d0164a05c admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:58.543132 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685eee1-cd81-49f0-8abb-0c5d0164a05c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:58.543414 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685eee1-cd81-49f0-8abb-0c5d0164a05c admin admin] PATH_INFO: `/v3/projects/8441bd006ded4a87a0f5db1d0266fcea/users/69f2c776784c44eda70bc707dfc11f2c/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:58.607787 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685eee1-cd81-49f0-8abb-0c5d0164a05c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8441bd006ded4a87a0f5db1d0266fcea, user_id=69f2c776784c44eda70bc707dfc11f2c, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:58.611150 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685eee1-cd81-49f0-8abb-0c5d0164a05c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:58.730937 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1162/2324] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:14:58 2022] PUT /identity/v3/projects/8441bd006ded4a87a0f5db1d0266fcea/users/69f2c776784c44eda70bc707dfc11f2c/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:58.733516 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d26bb5-1c5c-452b-bf8c-b5aafa22861d admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:58.746125 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d26bb5-1c5c-452b-bf8c-b5aafa22861d admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:58.778674 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:14:58.900521 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85f535-f020-4688-aa93-23e5e39dc2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:58.901719 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85f535-f020-4688-aa93-23e5e39dc2b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:58.903475 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85f535-f020-4688-aa93-23e5e39dc2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:58.904255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85f535-f020-4688-aa93-23e5e39dc2b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:58.905867 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e85f535-f020-4688-aa93-23e5e39dc2b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:58.909121 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e85f535-f020-4688-aa93-23e5e39dc2b5 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:58.924853 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1163/2325] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:14:58.928973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685eee1-cd81-49f0-8abb-0c5d0164a05c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:58.941580 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685eee1-cd81-49f0-8abb-0c5d0164a05c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:58.947970 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d036d8c-3954-4e9e-bb77-785aa22e2fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:58.950292 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d036d8c-3954-4e9e-bb77-785aa22e2fc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:58.951037 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d036d8c-3954-4e9e-bb77-785aa22e2fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:58.952787 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d036d8c-3954-4e9e-bb77-785aa22e2fc5 admin admin] PATH_INFO: `/v3/projects/8441bd006ded4a87a0f5db1d0266fcea/users/69f2c776784c44eda70bc707dfc11f2c/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:58.964304 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d036d8c-3954-4e9e-bb77-785aa22e2fc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8441bd006ded4a87a0f5db1d0266fcea, user_id=69f2c776784c44eda70bc707dfc11f2c, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:58.968396 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d036d8c-3954-4e9e-bb77-785aa22e2fc5 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:59.003651 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1163/2326] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:14:58 2022] PUT /identity/v3/projects/8441bd006ded4a87a0f5db1d0266fcea/users/69f2c776784c44eda70bc707dfc11f2c/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:14:59.003749 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e34f4-befe-4d20-8de9-c86490b1dbcb None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:59.003749 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e34f4-befe-4d20-8de9-c86490b1dbcb None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:59.003749 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e34f4-befe-4d20-8de9-c86490b1dbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:59.069598 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-2f2e34f4-befe-4d20-8de9-c86490b1dbcb None None] MFA Rules not processed for user `69f2c776784c44eda70bc707dfc11f2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:14:59.076329 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-2f2e34f4-befe-4d20-8de9-c86490b1dbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:59.277576 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1164/2327] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:14:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:14:59.490451 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d036d8c-3954-4e9e-bb77-785aa22e2fc5 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:14:59.491832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2d036d8c-3954-4e9e-bb77-785aa22e2fc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:14:59.699203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d036d8c-3954-4e9e-bb77-785aa22e2fc5 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:14:59.710406 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712655c-4710-42cf-931d-2bcd58fb0c09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:14:59.711063 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712655c-4710-42cf-931d-2bcd58fb0c09 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:14:59.711351 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712655c-4710-42cf-931d-2bcd58fb0c09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:14:59.711630 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712655c-4710-42cf-931d-2bcd58fb0c09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:14:59.745252 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a712655c-4710-42cf-931d-2bcd58fb0c09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:14:59.748442 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a712655c-4710-42cf-931d-2bcd58fb0c09 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:14:59.792542 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1164/2328] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:14:59 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:15:00.598595 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e34f4-befe-4d20-8de9-c86490b1dbcb None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:00.712225 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e34f4-befe-4d20-8de9-c86490b1dbcb None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:00.720560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1cb8b-39af-4358-944b-b21f5dbda71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:00.721522 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1cb8b-39af-4358-944b-b21f5dbda71b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:00.721991 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1cb8b-39af-4358-944b-b21f5dbda71b admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:00.722425 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1cb8b-39af-4358-944b-b21f5dbda71b admin admin] PATH_INFO: `/v3/users/407bc0300eac4db495e8bba5d6178a5e` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:00.724010 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb1cb8b-39af-4358-944b-b21f5dbda71b admin admin] RBAC: Authorizing `identity:delete_user(user_id=407bc0300eac4db495e8bba5d6178a5e)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:00.727332 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb1cb8b-39af-4358-944b-b21f5dbda71b admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:00.958415 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:15:00.962089 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-6bb1cb8b-39af-4358-944b-b21f5dbda71b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '407bc0300eac4db495e8bba5d6178a5e'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:01.043261 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-6bb1cb8b-39af-4358-944b-b21f5dbda71b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '407bc0300eac4db495e8bba5d6178a5e'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:01.044471 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-6bb1cb8b-39af-4358-944b-b21f5dbda71b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '407bc0300eac4db495e8bba5d6178a5e'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:01.110472 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-6bb1cb8b-39af-4358-944b-b21f5dbda71b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '407bc0300eac4db495e8bba5d6178a5e'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:01.114365 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-6bb1cb8b-39af-4358-944b-b21f5dbda71b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '407bc0300eac4db495e8bba5d6178a5e'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:01.145750 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1165/2329] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:15:00 2022] DELETE /identity/v3/users/407bc0300eac4db495e8bba5d6178a5e => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:15:01.152291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e295de-8930-4960-8dd4-df90221eaf01 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:01.152826 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e295de-8930-4960-8dd4-df90221eaf01 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:01.152826 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e295de-8930-4960-8dd4-df90221eaf01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:01.175461 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-64e295de-8930-4960-8dd4-df90221eaf01 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:01.176805 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-64e295de-8930-4960-8dd4-df90221eaf01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:01.260052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1165/2330] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:15:01.274096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1cb8b-39af-4358-944b-b21f5dbda71b admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:01.276740 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-6bb1cb8b-39af-4358-944b-b21f5dbda71b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:01.369420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1cb8b-39af-4358-944b-b21f5dbda71b admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:01.374127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7ca20-f007-4c22-a8fe-02c5fdd04a47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:01.385472 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7ca20-f007-4c22-a8fe-02c5fdd04a47 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:01.385472 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7ca20-f007-4c22-a8fe-02c5fdd04a47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:01.385472 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7ca20-f007-4c22-a8fe-02c5fdd04a47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:01.387491 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b7ca20-f007-4c22-a8fe-02c5fdd04a47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:01.388881 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b7ca20-f007-4c22-a8fe-02c5fdd04a47 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:01.400686 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1166/2331] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:15:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:15:03.161739 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e295de-8930-4960-8dd4-df90221eaf01 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:03.164152 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-64e295de-8930-4960-8dd4-df90221eaf01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:03.181196 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e295de-8930-4960-8dd4-df90221eaf01 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:03.189463 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ac2a4-e248-4208-8a18-795df8e81b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:03.190147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ac2a4-e248-4208-8a18-795df8e81b24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:03.190432 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ac2a4-e248-4208-8a18-795df8e81b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:03.190709 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ac2a4-e248-4208-8a18-795df8e81b24 admin admin] PATH_INFO: `/v3/projects/65c39ce41fb6438bb8abfc09000141c0` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:03.192997 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ac2a4-e248-4208-8a18-795df8e81b24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65c39ce41fb6438bb8abfc09000141c0)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:03.195792 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ac2a4-e248-4208-8a18-795df8e81b24 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:03.209594 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:15:03.359798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8d5ac2a4-e248-4208-8a18-795df8e81b24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65c39ce41fb6438bb8abfc09000141c0', 'request_id': 'req-8d5ac2a4-e248-4208-8a18-795df8e81b24'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:03.487339 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8d5ac2a4-e248-4208-8a18-795df8e81b24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65c39ce41fb6438bb8abfc09000141c0', 'request_id': 'req-8d5ac2a4-e248-4208-8a18-795df8e81b24'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:03.502105 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8d5ac2a4-e248-4208-8a18-795df8e81b24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65c39ce41fb6438bb8abfc09000141c0', 'request_id': 'req-8d5ac2a4-e248-4208-8a18-795df8e81b24'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:03.560464 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8d5ac2a4-e248-4208-8a18-795df8e81b24 admin admin] The token cache is being invalidate because project 65c39ce41fb6438bb8abfc09000141c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:15:03.561537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8d5ac2a4-e248-4208-8a18-795df8e81b24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:03.563745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8d5ac2a4-e248-4208-8a18-795df8e81b24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:03.565938 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1166/2332] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:15:03 2022] DELETE /identity/v3/projects/65c39ce41fb6438bb8abfc09000141c0 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:15:03.569816 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7ca20-f007-4c22-a8fe-02c5fdd04a47 service neutron] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:03.588331 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7ca20-f007-4c22-a8fe-02c5fdd04a47 service neutron] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:03.687896 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51344b5b-f736-4392-ae93-427a86288a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:03.688997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51344b5b-f736-4392-ae93-427a86288a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:03.689447 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51344b5b-f736-4392-ae93-427a86288a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:03.689907 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51344b5b-f736-4392-ae93-427a86288a8c admin admin] PATH_INFO: `/v3/users/69f2c776784c44eda70bc707dfc11f2c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:03.695003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51344b5b-f736-4392-ae93-427a86288a8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=69f2c776784c44eda70bc707dfc11f2c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:03.701159 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51344b5b-f736-4392-ae93-427a86288a8c admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:03.997520 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-51344b5b-f736-4392-ae93-427a86288a8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69f2c776784c44eda70bc707dfc11f2c'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:04.104797 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-51344b5b-f736-4392-ae93-427a86288a8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69f2c776784c44eda70bc707dfc11f2c'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:04.106719 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-51344b5b-f736-4392-ae93-427a86288a8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69f2c776784c44eda70bc707dfc11f2c'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:04.134702 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:15:04.174932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-51344b5b-f736-4392-ae93-427a86288a8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69f2c776784c44eda70bc707dfc11f2c'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:04.180959 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-51344b5b-f736-4392-ae93-427a86288a8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69f2c776784c44eda70bc707dfc11f2c'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:04.222544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1167/2333] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:15:03 2022] DELETE /identity/v3/users/69f2c776784c44eda70bc707dfc11f2c => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:15:04.941804 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ac2a4-e248-4208-8a18-795df8e81b24 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:04.943347 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8d5ac2a4-e248-4208-8a18-795df8e81b24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:04.976311 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ac2a4-e248-4208-8a18-795df8e81b24 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:05.133728 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6973f2-a40a-43a3-8fde-351af2e6efab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:05.134420 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6973f2-a40a-43a3-8fde-351af2e6efab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:05.134692 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6973f2-a40a-43a3-8fde-351af2e6efab admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:05.134997 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6973f2-a40a-43a3-8fde-351af2e6efab admin admin] PATH_INFO: `/v3/users/a4edc60562544cf6bd05847baf067deb` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:05.140346 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6973f2-a40a-43a3-8fde-351af2e6efab admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4edc60562544cf6bd05847baf067deb)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:05.143729 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6973f2-a40a-43a3-8fde-351af2e6efab admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:05.416213 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-af6973f2-a40a-43a3-8fde-351af2e6efab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4edc60562544cf6bd05847baf067deb'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:05.500170 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-af6973f2-a40a-43a3-8fde-351af2e6efab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4edc60562544cf6bd05847baf067deb'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:05.503531 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-af6973f2-a40a-43a3-8fde-351af2e6efab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4edc60562544cf6bd05847baf067deb'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:05.525886 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-af6973f2-a40a-43a3-8fde-351af2e6efab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4edc60562544cf6bd05847baf067deb'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:05.527313 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-af6973f2-a40a-43a3-8fde-351af2e6efab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4edc60562544cf6bd05847baf067deb'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:05.554335 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1167/2334] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:15:04 2022] DELETE /identity/v3/users/a4edc60562544cf6bd05847baf067deb => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:15:05.560166 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9698eb-a78c-4393-b487-94ee06f9d593 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:05.560730 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9698eb-a78c-4393-b487-94ee06f9d593 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:05.560730 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9698eb-a78c-4393-b487-94ee06f9d593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:05.617289 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-af9698eb-a78c-4393-b487-94ee06f9d593 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:05.619023 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-af9698eb-a78c-4393-b487-94ee06f9d593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:05.812112 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1168/2335] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:15:05.877951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6973f2-a40a-43a3-8fde-351af2e6efab admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:05.879446 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-af6973f2-a40a-43a3-8fde-351af2e6efab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:05.924354 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:15:05.977263 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6973f2-a40a-43a3-8fde-351af2e6efab admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:05.983525 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d4faa-137a-405e-a5f9-d7abc57ad418 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:05.985106 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d4faa-137a-405e-a5f9-d7abc57ad418 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:05.985739 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d4faa-137a-405e-a5f9-d7abc57ad418 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:05.986174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d4faa-137a-405e-a5f9-d7abc57ad418 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:05.998263 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d4faa-137a-405e-a5f9-d7abc57ad418 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:06.000294 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d4faa-137a-405e-a5f9-d7abc57ad418 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:06.013246 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1168/2336] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:15:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:15:06.140720 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9698eb-a78c-4393-b487-94ee06f9d593 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:06.142784 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-af9698eb-a78c-4393-b487-94ee06f9d593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:06.167025 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9698eb-a78c-4393-b487-94ee06f9d593 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:06.191245 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868bf822-7f51-468e-9bd2-0f676b937a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:06.193159 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868bf822-7f51-468e-9bd2-0f676b937a71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:06.193828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868bf822-7f51-468e-9bd2-0f676b937a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:06.196587 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868bf822-7f51-468e-9bd2-0f676b937a71 admin admin] PATH_INFO: `/v3/projects/8441bd006ded4a87a0f5db1d0266fcea` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:06.199972 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868bf822-7f51-468e-9bd2-0f676b937a71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8441bd006ded4a87a0f5db1d0266fcea)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:06.208843 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868bf822-7f51-468e-9bd2-0f676b937a71 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:06.315425 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b808de-1d98-40c1-8809-0d70bad2bde4 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:06.315828 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b808de-1d98-40c1-8809-0d70bad2bde4 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:06.316678 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b808de-1d98-40c1-8809-0d70bad2bde4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:06.359169 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-52b808de-1d98-40c1-8809-0d70bad2bde4 None None] MFA Rules not processed for user `de853a2dfffb4f05b6f47df2280ae4e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:06.360619 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-52b808de-1d98-40c1-8809-0d70bad2bde4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:06.450396 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-868bf822-7f51-468e-9bd2-0f676b937a71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8441bd006ded4a87a0f5db1d0266fcea', 'request_id': 'req-868bf822-7f51-468e-9bd2-0f676b937a71'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:06.465492 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-868bf822-7f51-468e-9bd2-0f676b937a71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8441bd006ded4a87a0f5db1d0266fcea', 'request_id': 'req-868bf822-7f51-468e-9bd2-0f676b937a71'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:06.518777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1169/2337] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Aug 3 05:15:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:15:06.549060 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-868bf822-7f51-468e-9bd2-0f676b937a71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8441bd006ded4a87a0f5db1d0266fcea', 'request_id': 'req-868bf822-7f51-468e-9bd2-0f676b937a71'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:06.588317 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-868bf822-7f51-468e-9bd2-0f676b937a71 admin admin] The token cache is being invalidate because project 8441bd006ded4a87a0f5db1d0266fcea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:15:06.590700 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-868bf822-7f51-468e-9bd2-0f676b937a71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:06.592843 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-868bf822-7f51-468e-9bd2-0f676b937a71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:06.595711 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b808de-1d98-40c1-8809-0d70bad2bde4 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:06.597260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-52b808de-1d98-40c1-8809-0d70bad2bde4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:06.603645 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1169/2338] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:15:06 2022] DELETE /identity/v3/projects/8441bd006ded4a87a0f5db1d0266fcea => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:15:06.614552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b6452-b0cc-42c7-afe6-3d9e26c537da None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:06.614884 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b6452-b0cc-42c7-afe6-3d9e26c537da None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:06.617335 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b6452-b0cc-42c7-afe6-3d9e26c537da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:06.691804 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-261b6452-b0cc-42c7-afe6-3d9e26c537da None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:06.702625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-261b6452-b0cc-42c7-afe6-3d9e26c537da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:06.824382 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b808de-1d98-40c1-8809-0d70bad2bde4 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:06.865634 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1170/2339] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:15:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:15:06.872848 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229cc49-37b0-43eb-984c-1d4392c5aa7e None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:06.872848 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229cc49-37b0-43eb-984c-1d4392c5aa7e None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:06.873842 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229cc49-37b0-43eb-984c-1d4392c5aa7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:06.909294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:15:06.922466 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-e229cc49-37b0-43eb-984c-1d4392c5aa7e None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:06.923947 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-e229cc49-37b0-43eb-984c-1d4392c5aa7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:06.939792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1171/2340] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:15:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:15:06.944121 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e229cc49-37b0-43eb-984c-1d4392c5aa7e None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:06.946355 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc5d993-7cd9-447d-99ae-7f6e3071ed09 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1014bcb4d36441aa46042941c9a63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:06.947098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5d993-7cd9-447d-99ae-7f6e3071ed09 service swift] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:06.947326 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5d993-7cd9-447d-99ae-7f6e3071ed09 service swift] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:06.947607 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5d993-7cd9-447d-99ae-7f6e3071ed09 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:06.952778 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2bc5d993-7cd9-447d-99ae-7f6e3071ed09 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:06.957303 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e229cc49-37b0-43eb-984c-1d4392c5aa7e None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:06.963855 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88409b67-401c-4019-b951-e060f8c49f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:06.964814 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88409b67-401c-4019-b951-e060f8c49f16 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:06.965206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88409b67-401c-4019-b951-e060f8c49f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:06.965609 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88409b67-401c-4019-b951-e060f8c49f16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:06.966492 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88409b67-401c-4019-b951-e060f8c49f16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:15:06.967238 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88409b67-401c-4019-b951-e060f8c49f16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:06.969465 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88409b67-401c-4019-b951-e060f8c49f16 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:06.988086 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1172/2341] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:15:06.996092 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88409b67-401c-4019-b951-e060f8c49f16 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:07.047408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc5d993-7cd9-447d-99ae-7f6e3071ed09 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:07.051059 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc5d993-7cd9-447d-99ae-7f6e3071ed09 service swift] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:07.057406 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88409b67-401c-4019-b951-e060f8c49f16 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:07.063832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5595078-a625-43d4-9b0f-77fb51171a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:07.065609 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5595078-a625-43d4-9b0f-77fb51171a16 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:07.066144 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5595078-a625-43d4-9b0f-77fb51171a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:07.070993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5595078-a625-43d4-9b0f-77fb51171a16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:07.072107 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5595078-a625-43d4-9b0f-77fb51171a16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:07.074591 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5595078-a625-43d4-9b0f-77fb51171a16 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:07.104680 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1170/2342] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 3 05:15:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 05:15:07.243734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1173/2343] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:15:06 2022] POST /identity/v3/projects => generated 377 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:15:07.249991 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc5d993-7cd9-447d-99ae-7f6e3071ed09 service swift] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:07.272744 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc5d993-7cd9-447d-99ae-7f6e3071ed09 service swift] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:07.527718 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:15:07.572235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d510ffed-2c46-4e38-8eab-3b9d502ce3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:07.573929 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510ffed-2c46-4e38-8eab-3b9d502ce3ae admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:07.574339 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510ffed-2c46-4e38-8eab-3b9d502ce3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:07.574910 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510ffed-2c46-4e38-8eab-3b9d502ce3ae admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:07.577050 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d510ffed-2c46-4e38-8eab-3b9d502ce3ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:07.579851 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d510ffed-2c46-4e38-8eab-3b9d502ce3ae admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:07.716000 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1171/2344] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:15:07 2022] POST /identity/v3/users => generated 289 bytes in 468 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:15:07.721419 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5595078-a625-43d4-9b0f-77fb51171a16 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:07.742378 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5595078-a625-43d4-9b0f-77fb51171a16 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:07.757470 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6541e3-02d2-4daf-8392-3c1c2f847146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:07.764898 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6541e3-02d2-4daf-8392-3c1c2f847146 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:07.765644 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6541e3-02d2-4daf-8392-3c1c2f847146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:07.766318 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6541e3-02d2-4daf-8392-3c1c2f847146 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:07.769115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6541e3-02d2-4daf-8392-3c1c2f847146 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:07.773076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6541e3-02d2-4daf-8392-3c1c2f847146 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:07.838496 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1174/2345] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:15:07.843216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d510ffed-2c46-4e38-8eab-3b9d502ce3ae admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:07.860727 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d510ffed-2c46-4e38-8eab-3b9d502ce3ae admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:07.877493 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96d848-6a73-4d45-9d81-ef65619f9440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:07.878222 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96d848-6a73-4d45-9d81-ef65619f9440 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:07.878514 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96d848-6a73-4d45-9d81-ef65619f9440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:07.878794 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96d848-6a73-4d45-9d81-ef65619f9440 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:07.879566 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa96d848-6a73-4d45-9d81-ef65619f9440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:07.892257 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa96d848-6a73-4d45-9d81-ef65619f9440 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:07.926563 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1172/2346] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:15:07.930340 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6541e3-02d2-4daf-8392-3c1c2f847146 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:07.946725 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6541e3-02d2-4daf-8392-3c1c2f847146 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:07.989138 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30229a-1965-47c2-8a62-39cde8ea1853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:07.991283 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30229a-1965-47c2-8a62-39cde8ea1853 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:07.995221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30229a-1965-47c2-8a62-39cde8ea1853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:07.995655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30229a-1965-47c2-8a62-39cde8ea1853 admin admin] PATH_INFO: `/v3/projects/ff2ad6000c204b4293ec1dec28763781/users/e50fd6561a584c6e90bbf51a3ee0fec9/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:08.041692 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da30229a-1965-47c2-8a62-39cde8ea1853 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff2ad6000c204b4293ec1dec28763781, user_id=e50fd6561a584c6e90bbf51a3ee0fec9, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:08.047168 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da30229a-1965-47c2-8a62-39cde8ea1853 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:08.161347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1175/2347] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:15:07 2022] PUT /identity/v3/projects/ff2ad6000c204b4293ec1dec28763781/users/e50fd6561a584c6e90bbf51a3ee0fec9/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:15:08.174764 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8e475-9d96-4c87-982a-53f2ea88c595 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:08.175648 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8e475-9d96-4c87-982a-53f2ea88c595 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:08.176079 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8e475-9d96-4c87-982a-53f2ea88c595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:08.252650 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-1ff8e475-9d96-4c87-982a-53f2ea88c595 None None] MFA Rules not processed for user `e50fd6561a584c6e90bbf51a3ee0fec9`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:08.254340 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1ff8e475-9d96-4c87-982a-53f2ea88c595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:08.477046 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd0a7f-50f3-4685-8149-11dfea6603b2 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:08.477636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd0a7f-50f3-4685-8149-11dfea6603b2 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:08.477939 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd0a7f-50f3-4685-8149-11dfea6603b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:08.489037 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1173/2348] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:15:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:15:08.504033 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff8e475-9d96-4c87-982a-53f2ea88c595 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:08.509925 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1ff8e475-9d96-4c87-982a-53f2ea88c595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:08.568632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:15:08.582878 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-25cd0a7f-50f3-4685-8149-11dfea6603b2 None None] MFA Rules not processed for user `7a806f98beb5459085fd4cca1f28da96`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:08.588821 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-25cd0a7f-50f3-4685-8149-11dfea6603b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:08.708904 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff8e475-9d96-4c87-982a-53f2ea88c595 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:08.720854 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a64c0-1913-444f-93e5-fe749d1f2f81 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cc14b33895443b893fab487bfe1858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:08.722541 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a64c0-1913-444f-93e5-fe749d1f2f81 service glance] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:08.723027 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a64c0-1913-444f-93e5-fe749d1f2f81 service glance] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:08.723457 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a64c0-1913-444f-93e5-fe749d1f2f81 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:08.726187 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1176/2349] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:15:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:15:08.727226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd0a7f-50f3-4685-8149-11dfea6603b2 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:08.729234 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-25cd0a7f-50f3-4685-8149-11dfea6603b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:08.747533 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9a64c0-1913-444f-93e5-fe749d1f2f81 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:08.749105 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9a64c0-1913-444f-93e5-fe749d1f2f81 service glance] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:08.782481 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1174/2350] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:15:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:15:08.860850 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd0a7f-50f3-4685-8149-11dfea6603b2 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:08.869415 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ccf3e-d8be-444d-be6b-f5f6a28030d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:08.870094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ccf3e-d8be-444d-be6b-f5f6a28030d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:08.870390 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ccf3e-d8be-444d-be6b-f5f6a28030d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:08.871038 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ccf3e-d8be-444d-be6b-f5f6a28030d0 admin admin] PATH_INFO: `/v3/projects/973189f7b53a4b86a32d72eb6010a3fd` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:08.871912 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ccf3e-d8be-444d-be6b-f5f6a28030d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=973189f7b53a4b86a32d72eb6010a3fd)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:08.874885 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ccf3e-d8be-444d-be6b-f5f6a28030d0 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:08.895940 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:15:09.028771 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-391ccf3e-d8be-444d-be6b-f5f6a28030d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '973189f7b53a4b86a32d72eb6010a3fd', 'request_id': 'req-391ccf3e-d8be-444d-be6b-f5f6a28030d0'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:09.036876 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-391ccf3e-d8be-444d-be6b-f5f6a28030d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '973189f7b53a4b86a32d72eb6010a3fd', 'request_id': 'req-391ccf3e-d8be-444d-be6b-f5f6a28030d0'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:09.138498 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-391ccf3e-d8be-444d-be6b-f5f6a28030d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '973189f7b53a4b86a32d72eb6010a3fd', 'request_id': 'req-391ccf3e-d8be-444d-be6b-f5f6a28030d0'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:09.145991 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a64c0-1913-444f-93e5-fe749d1f2f81 service glance] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:09.155308 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8b9a64c0-1913-444f-93e5-fe749d1f2f81 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:09.158509 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-391ccf3e-d8be-444d-be6b-f5f6a28030d0 admin admin] The token cache is being invalidate because project 973189f7b53a4b86a32d72eb6010a3fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:15:09.158806 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-391ccf3e-d8be-444d-be6b-f5f6a28030d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:09.161354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-391ccf3e-d8be-444d-be6b-f5f6a28030d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:09.177464 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1177/2351] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:15:08 2022] DELETE /identity/v3/projects/973189f7b53a4b86a32d72eb6010a3fd => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:15:09.426008 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a64c0-1913-444f-93e5-fe749d1f2f81 service glance] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:09.617946 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1410006-c58b-4d0f-a1c9-9ba1c2ae2e55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:09.623883 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1410006-c58b-4d0f-a1c9-9ba1c2ae2e55 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:09.624528 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1410006-c58b-4d0f-a1c9-9ba1c2ae2e55 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:09.625013 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1410006-c58b-4d0f-a1c9-9ba1c2ae2e55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:09.635109 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c1410006-c58b-4d0f-a1c9-9ba1c2ae2e55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:09.805189 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:15:09.808163 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1410006-c58b-4d0f-a1c9-9ba1c2ae2e55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:09.809825 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1410006-c58b-4d0f-a1c9-9ba1c2ae2e55 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:09.870216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1175/2352] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:15:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 725 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:15:11.691991 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62550691-2f80-4bee-9287-d1e3a518a23f None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:11.691991 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62550691-2f80-4bee-9287-d1e3a518a23f None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:11.691991 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62550691-2f80-4bee-9287-d1e3a518a23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:11.769536 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-62550691-2f80-4bee-9287-d1e3a518a23f None None] MFA Rules not processed for user `79539af2999d48baa11734ced32dccb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:11.770902 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-62550691-2f80-4bee-9287-d1e3a518a23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:11.937218 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1178/2353] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:15:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:15:11.938093 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1410006-c58b-4d0f-a1c9-9ba1c2ae2e55 service nova] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:11.938093 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c1410006-c58b-4d0f-a1c9-9ba1c2ae2e55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:12.047051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1410006-c58b-4d0f-a1c9-9ba1c2ae2e55 service nova] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:12.053082 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebd2f8-ad26-4aaa-8207-2e59d14392e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:12.053082 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebd2f8-ad26-4aaa-8207-2e59d14392e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:12.053082 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebd2f8-ad26-4aaa-8207-2e59d14392e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:12.053433 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebd2f8-ad26-4aaa-8207-2e59d14392e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:12.053984 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ebd2f8-ad26-4aaa-8207-2e59d14392e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:12.057863 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ebd2f8-ad26-4aaa-8207-2e59d14392e1 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:12.385290 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1176/2354] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:15:11 2022] POST /identity/v3/projects => generated 383 bytes in 464 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:15:12.390144 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62550691-2f80-4bee-9287-d1e3a518a23f None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:12.407457 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62550691-2f80-4bee-9287-d1e3a518a23f None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:12.667910 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ccb78c-9352-41ad-84f6-eb8e4233dc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:12.668572 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccb78c-9352-41ad-84f6-eb8e4233dc81 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:12.668863 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccb78c-9352-41ad-84f6-eb8e4233dc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:12.669145 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccb78c-9352-41ad-84f6-eb8e4233dc81 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:12.670392 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ccb78c-9352-41ad-84f6-eb8e4233dc81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:12.673211 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ccb78c-9352-41ad-84f6-eb8e4233dc81 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:12.803665 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1179/2355] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:15:12 2022] POST /identity/v3/users => generated 298 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:15:12.809565 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebd2f8-ad26-4aaa-8207-2e59d14392e1 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:12.823817 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebd2f8-ad26-4aaa-8207-2e59d14392e1 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:12.858221 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e4f40-e21e-4e71-8e39-fa862e632849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:12.858913 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e4f40-e21e-4e71-8e39-fa862e632849 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:12.859207 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e4f40-e21e-4e71-8e39-fa862e632849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:12.859488 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e4f40-e21e-4e71-8e39-fa862e632849 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:12.861774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e4f40-e21e-4e71-8e39-fa862e632849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:12.866656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e4f40-e21e-4e71-8e39-fa862e632849 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:12.899007 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1177/2356] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:15:12.902602 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ccb78c-9352-41ad-84f6-eb8e4233dc81 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:12.933586 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ccb78c-9352-41ad-84f6-eb8e4233dc81 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:12.940787 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b0885-25cc-4629-b1a2-438e76647a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:12.941505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b0885-25cc-4629-b1a2-438e76647a3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:12.941793 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b0885-25cc-4629-b1a2-438e76647a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:12.942454 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b0885-25cc-4629-b1a2-438e76647a3c admin admin] PATH_INFO: `/v3/projects/f0602692bf964a639462a788ecfd052c/users/9f01ab5bffd74ecfa11c9cec030a0dd3/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:12.987308 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3b0885-25cc-4629-b1a2-438e76647a3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0602692bf964a639462a788ecfd052c, user_id=9f01ab5bffd74ecfa11c9cec030a0dd3, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:12.991330 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3b0885-25cc-4629-b1a2-438e76647a3c admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:13.085900 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:15:13.089567 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1180/2357] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:15:12 2022] PUT /identity/v3/projects/f0602692bf964a639462a788ecfd052c/users/9f01ab5bffd74ecfa11c9cec030a0dd3/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:15:13.094722 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e4f40-e21e-4e71-8e39-fa862e632849 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:13.109665 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e4f40-e21e-4e71-8e39-fa862e632849 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:13.320075 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab79453c-ddc5-46f4-8f67-d025cfd3f7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:13.320804 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79453c-ddc5-46f4-8f67-d025cfd3f7fc admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:13.321091 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79453c-ddc5-46f4-8f67-d025cfd3f7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:13.321376 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79453c-ddc5-46f4-8f67-d025cfd3f7fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:13.322071 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab79453c-ddc5-46f4-8f67-d025cfd3f7fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:13.333523 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab79453c-ddc5-46f4-8f67-d025cfd3f7fc admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:13.380042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b0885-25cc-4629-b1a2-438e76647a3c admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:13.382893 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1178/2358] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:15:13.407506 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b0885-25cc-4629-b1a2-438e76647a3c admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:13.414229 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3623dc-7711-4f48-96f3-8f2070ee6be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:13.415333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3623dc-7711-4f48-96f3-8f2070ee6be3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:13.415910 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3623dc-7711-4f48-96f3-8f2070ee6be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:13.416665 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3623dc-7711-4f48-96f3-8f2070ee6be3 admin admin] PATH_INFO: `/v3/projects/f0602692bf964a639462a788ecfd052c/users/9f01ab5bffd74ecfa11c9cec030a0dd3/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:13.419484 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3623dc-7711-4f48-96f3-8f2070ee6be3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0602692bf964a639462a788ecfd052c, user_id=9f01ab5bffd74ecfa11c9cec030a0dd3, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:13.423352 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3623dc-7711-4f48-96f3-8f2070ee6be3 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:13.443556 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1181/2359] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:15:13 2022] PUT /identity/v3/projects/f0602692bf964a639462a788ecfd052c/users/9f01ab5bffd74ecfa11c9cec030a0dd3/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:15:15.824325 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19b334-2dfc-4b80-a694-9a1f4932e4b0 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:15.825238 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19b334-2dfc-4b80-a694-9a1f4932e4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:15.825670 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19b334-2dfc-4b80-a694-9a1f4932e4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:15.885156 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-5c19b334-2dfc-4b80-a694-9a1f4932e4b0 None None] MFA Rules not processed for user `f145cffb3c4d4965a35553b95b316f56`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:15.886756 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-5c19b334-2dfc-4b80-a694-9a1f4932e4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:16.009517 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1179/2360] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:15:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:15:16.505562 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012c3ad-2423-47e1-9938-95012b50e70a None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:16.506083 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012c3ad-2423-47e1-9938-95012b50e70a None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:16.506173 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012c3ad-2423-47e1-9938-95012b50e70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:16.539526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-5012c3ad-2423-47e1-9938-95012b50e70a None None] MFA Rules not processed for user `bb0fe4cb98e0447cad3e126da1a8de9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:16.541910 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-5012c3ad-2423-47e1-9938-95012b50e70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:16.677800 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1182/2361] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:15:16 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:15:23.665730 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69cca65-4a43-4bd5-8bd4-5588c9a72af6 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:23.666812 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69cca65-4a43-4bd5-8bd4-5588c9a72af6 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:23.667286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69cca65-4a43-4bd5-8bd4-5588c9a72af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:23.735830 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:15:23.782106 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-c69cca65-4a43-4bd5-8bd4-5588c9a72af6 None None] MFA Rules not processed for user `9f01ab5bffd74ecfa11c9cec030a0dd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:23.783969 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c69cca65-4a43-4bd5-8bd4-5588c9a72af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:23.975598 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1180/2362] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:15:23 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:15:24.012966 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012c3ad-2423-47e1-9938-95012b50e70a None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:24.015954 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-5012c3ad-2423-47e1-9938-95012b50e70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:24.178975 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012c3ad-2423-47e1-9938-95012b50e70a None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:24.188890 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4daa00c-e529-41d7-9bcc-8ab95781dd6e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cc14b33895443b893fab487bfe1858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:24.190011 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4daa00c-e529-41d7-9bcc-8ab95781dd6e service glance] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:24.191269 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4daa00c-e529-41d7-9bcc-8ab95781dd6e service glance] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:24.195495 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4daa00c-e529-41d7-9bcc-8ab95781dd6e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:24.319755 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4daa00c-e529-41d7-9bcc-8ab95781dd6e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:24.321213 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4daa00c-e529-41d7-9bcc-8ab95781dd6e service glance] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:24.340600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1183/2363] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:15:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:15:24.728304 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ed542-0aaf-453e-bc65-8f5d8e431b43 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:24.730225 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ed542-0aaf-453e-bc65-8f5d8e431b43 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:24.730225 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ed542-0aaf-453e-bc65-8f5d8e431b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:24.750783 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-5a7ed542-0aaf-453e-bc65-8f5d8e431b43 None None] MFA Rules not processed for user `de853a2dfffb4f05b6f47df2280ae4e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:24.753370 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-5a7ed542-0aaf-453e-bc65-8f5d8e431b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:24.911020 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1181/2364] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Aug 3 05:15:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:15:24.936731 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4daa00c-e529-41d7-9bcc-8ab95781dd6e service glance] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:24.938389 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-e4daa00c-e529-41d7-9bcc-8ab95781dd6e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:25.055505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:15:25.101492 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4daa00c-e529-41d7-9bcc-8ab95781dd6e service glance] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:25.108271 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e3813c-e40e-4ee4-bc95-1f14ae7edfbb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1014bcb4d36441aa46042941c9a63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:25.109207 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3813c-e40e-4ee4-bc95-1f14ae7edfbb service swift] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:25.110480 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3813c-e40e-4ee4-bc95-1f14ae7edfbb service swift] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:25.110985 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3813c-e40e-4ee4-bc95-1f14ae7edfbb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:25.124434 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e3813c-e40e-4ee4-bc95-1f14ae7edfbb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:25.125905 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e3813c-e40e-4ee4-bc95-1f14ae7edfbb service swift] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:25.136361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1184/2365] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 3 05:15:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 05:15:28.686424 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d14ed-1586-4e3d-a723-36cdd3b59815 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:28.687135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d14ed-1586-4e3d-a723-36cdd3b59815 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:28.687135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d14ed-1586-4e3d-a723-36cdd3b59815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:28.718409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-226d14ed-1586-4e3d-a723-36cdd3b59815 None None] MFA Rules not processed for user `de853a2dfffb4f05b6f47df2280ae4e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:28.719713 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-226d14ed-1586-4e3d-a723-36cdd3b59815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:28.729372 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1182/2366] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Aug 3 05:15:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:15:28.792638 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e3813c-e40e-4ee4-bc95-1f14ae7edfbb service swift] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:28.837764 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e3813c-e40e-4ee4-bc95-1f14ae7edfbb service swift] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:28.848554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95486e-c110-460d-974f-3f74370063fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1014bcb4d36441aa46042941c9a63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:28.859178 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95486e-c110-460d-974f-3f74370063fe service swift] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:28.861809 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95486e-c110-460d-974f-3f74370063fe service swift] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:28.861809 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95486e-c110-460d-974f-3f74370063fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:28.873491 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f95486e-c110-460d-974f-3f74370063fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:28.875433 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f95486e-c110-460d-974f-3f74370063fe service swift] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:28.885337 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1185/2367] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 3 05:15:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 05:15:34.913655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d14ed-1586-4e3d-a723-36cdd3b59815 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:34.915850 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-226d14ed-1586-4e3d-a723-36cdd3b59815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:35.027634 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d14ed-1586-4e3d-a723-36cdd3b59815 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:35.034213 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0c4bc-b153-4777-8f13-a45ebc5db6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:35.034933 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0c4bc-b153-4777-8f13-a45ebc5db6b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:35.035223 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0c4bc-b153-4777-8f13-a45ebc5db6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:35.035509 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0c4bc-b153-4777-8f13-a45ebc5db6b2 admin admin] PATH_INFO: `/v3/users/e50fd6561a584c6e90bbf51a3ee0fec9` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:35.036730 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0c4bc-b153-4777-8f13-a45ebc5db6b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e50fd6561a584c6e90bbf51a3ee0fec9)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:35.038689 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0c4bc-b153-4777-8f13-a45ebc5db6b2 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:35.479454 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:15:35.502789 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-27d0c4bc-b153-4777-8f13-a45ebc5db6b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e50fd6561a584c6e90bbf51a3ee0fec9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:35.594741 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-27d0c4bc-b153-4777-8f13-a45ebc5db6b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e50fd6561a584c6e90bbf51a3ee0fec9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:35.595718 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-27d0c4bc-b153-4777-8f13-a45ebc5db6b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e50fd6561a584c6e90bbf51a3ee0fec9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:35.612414 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-27d0c4bc-b153-4777-8f13-a45ebc5db6b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e50fd6561a584c6e90bbf51a3ee0fec9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:35.613430 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-27d0c4bc-b153-4777-8f13-a45ebc5db6b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e50fd6561a584c6e90bbf51a3ee0fec9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:35.629270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1183/2368] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:15:34 2022] DELETE /identity/v3/users/e50fd6561a584c6e90bbf51a3ee0fec9 => generated 0 bytes in 716 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:15:35.633419 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c915b1d-afd3-44d1-a476-b9162dcff1b3 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:35.634295 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c915b1d-afd3-44d1-a476-b9162dcff1b3 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:35.634735 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c915b1d-afd3-44d1-a476-b9162dcff1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:35.677554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-0c915b1d-afd3-44d1-a476-b9162dcff1b3 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:35.683482 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-0c915b1d-afd3-44d1-a476-b9162dcff1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:35.769790 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1186/2369] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:15:35.781439 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0c4bc-b153-4777-8f13-a45ebc5db6b2 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:35.783669 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-27d0c4bc-b153-4777-8f13-a45ebc5db6b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:35.960379 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0c4bc-b153-4777-8f13-a45ebc5db6b2 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:35.970821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd989f1-7b27-4c49-9331-7c5f15b4ce14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:35.971768 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd989f1-7b27-4c49-9331-7c5f15b4ce14 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:35.972214 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd989f1-7b27-4c49-9331-7c5f15b4ce14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:35.972667 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd989f1-7b27-4c49-9331-7c5f15b4ce14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:36.017323 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd989f1-7b27-4c49-9331-7c5f15b4ce14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:36.018993 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd989f1-7b27-4c49-9331-7c5f15b4ce14 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:36.042300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1184/2370] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:15:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:15:36.839030 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c915b1d-afd3-44d1-a476-b9162dcff1b3 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:36.840650 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-0c915b1d-afd3-44d1-a476-b9162dcff1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:36.875285 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c915b1d-afd3-44d1-a476-b9162dcff1b3 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:36.880601 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d9557-6ba0-40b3-ac74-633ad55d2c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:36.881263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d9557-6ba0-40b3-ac74-633ad55d2c99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:36.881539 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d9557-6ba0-40b3-ac74-633ad55d2c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:36.881814 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d9557-6ba0-40b3-ac74-633ad55d2c99 admin admin] PATH_INFO: `/v3/projects/ff2ad6000c204b4293ec1dec28763781` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:36.884298 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4d9557-6ba0-40b3-ac74-633ad55d2c99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff2ad6000c204b4293ec1dec28763781)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:36.886238 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4d9557-6ba0-40b3-ac74-633ad55d2c99 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:37.029603 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-8f4d9557-6ba0-40b3-ac74-633ad55d2c99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff2ad6000c204b4293ec1dec28763781', 'request_id': 'req-8f4d9557-6ba0-40b3-ac74-633ad55d2c99'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:37.042561 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-8f4d9557-6ba0-40b3-ac74-633ad55d2c99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff2ad6000c204b4293ec1dec28763781', 'request_id': 'req-8f4d9557-6ba0-40b3-ac74-633ad55d2c99'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:37.148026 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-8f4d9557-6ba0-40b3-ac74-633ad55d2c99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff2ad6000c204b4293ec1dec28763781', 'request_id': 'req-8f4d9557-6ba0-40b3-ac74-633ad55d2c99'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:37.165551 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-8f4d9557-6ba0-40b3-ac74-633ad55d2c99 admin admin] The token cache is being invalidate because project ff2ad6000c204b4293ec1dec28763781 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:15:37.165844 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-8f4d9557-6ba0-40b3-ac74-633ad55d2c99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:37.168150 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-8f4d9557-6ba0-40b3-ac74-633ad55d2c99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:37.172926 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1187/2371] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:15:36 2022] DELETE /identity/v3/projects/ff2ad6000c204b4293ec1dec28763781 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:15:37.178242 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8dc4c7-6b6d-49ce-b34b-6db29ecdcc03 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:37.178767 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8dc4c7-6b6d-49ce-b34b-6db29ecdcc03 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:37.179074 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8dc4c7-6b6d-49ce-b34b-6db29ecdcc03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:37.212281 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-7f8dc4c7-6b6d-49ce-b34b-6db29ecdcc03 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:37.216712 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7f8dc4c7-6b6d-49ce-b34b-6db29ecdcc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:37.335225 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1185/2372] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:15:37.338906 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a169ad6-0846-41d4-8e7c-d6e2fb98fc94 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:37.338906 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a169ad6-0846-41d4-8e7c-d6e2fb98fc94 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:37.338906 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a169ad6-0846-41d4-8e7c-d6e2fb98fc94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:37.378947 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-0a169ad6-0846-41d4-8e7c-d6e2fb98fc94 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:37.380328 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-0a169ad6-0846-41d4-8e7c-d6e2fb98fc94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:37.401610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1188/2373] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:15:37.401816 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8dc4c7-6b6d-49ce-b34b-6db29ecdcc03 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:37.414767 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8dc4c7-6b6d-49ce-b34b-6db29ecdcc03 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:37.457741 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a4977-b9a1-4503-bafb-44308c0addba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:37.458414 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a4977-b9a1-4503-bafb-44308c0addba admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:37.458781 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a4977-b9a1-4503-bafb-44308c0addba admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:37.460356 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a4977-b9a1-4503-bafb-44308c0addba admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:37.461294 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51a4977-b9a1-4503-bafb-44308c0addba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:15:37.472490 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51a4977-b9a1-4503-bafb-44308c0addba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:37.474356 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51a4977-b9a1-4503-bafb-44308c0addba admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:37.543720 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1186/2374] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:15:37.555366 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a169ad6-0846-41d4-8e7c-d6e2fb98fc94 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:37.570020 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a169ad6-0846-41d4-8e7c-d6e2fb98fc94 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:37.588402 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d6ebf-4155-49d5-a88a-c50249e310b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:37.589150 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d6ebf-4155-49d5-a88a-c50249e310b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:37.589529 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d6ebf-4155-49d5-a88a-c50249e310b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:37.589882 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d6ebf-4155-49d5-a88a-c50249e310b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:37.590680 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d6ebf-4155-49d5-a88a-c50249e310b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:37.593094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d6ebf-4155-49d5-a88a-c50249e310b7 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:37.730869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1189/2375] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:15:37 2022] POST /identity/v3/projects => generated 377 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:15:37.735491 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a4977-b9a1-4503-bafb-44308c0addba admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:37.766594 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a4977-b9a1-4503-bafb-44308c0addba admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:37.844509 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:15:37.874236 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d0b2a-bb8a-4394-b397-334ead993856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:37.875188 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d0b2a-bb8a-4394-b397-334ead993856 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:37.875701 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d0b2a-bb8a-4394-b397-334ead993856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:37.876158 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d0b2a-bb8a-4394-b397-334ead993856 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:37.877023 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d0b2a-bb8a-4394-b397-334ead993856 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:37.881241 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d0b2a-bb8a-4394-b397-334ead993856 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:37.997897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1187/2376] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:15:37 2022] POST /identity/v3/users => generated 289 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:15:37.999475 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d6ebf-4155-49d5-a88a-c50249e310b7 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:38.021010 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d6ebf-4155-49d5-a88a-c50249e310b7 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:38.050352 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424b2f7-0c5f-4ddd-a0b5-2ab2ac20de61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:38.051333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424b2f7-0c5f-4ddd-a0b5-2ab2ac20de61 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:38.051775 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424b2f7-0c5f-4ddd-a0b5-2ab2ac20de61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:38.052232 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424b2f7-0c5f-4ddd-a0b5-2ab2ac20de61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:38.053138 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5424b2f7-0c5f-4ddd-a0b5-2ab2ac20de61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:38.055083 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5424b2f7-0c5f-4ddd-a0b5-2ab2ac20de61 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:38.106575 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d0b2a-bb8a-4394-b397-334ead993856 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:38.110293 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1190/2377] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:15:38.125626 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d0b2a-bb8a-4394-b397-334ead993856 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:38.135543 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d048071-f2c3-43f0-b4d3-f42cd2700351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:38.136244 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d048071-f2c3-43f0-b4d3-f42cd2700351 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:38.136530 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d048071-f2c3-43f0-b4d3-f42cd2700351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:38.136808 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d048071-f2c3-43f0-b4d3-f42cd2700351 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:38.139887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d048071-f2c3-43f0-b4d3-f42cd2700351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:38.146877 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d048071-f2c3-43f0-b4d3-f42cd2700351 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:38.180769 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1188/2378] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:15:38.182978 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424b2f7-0c5f-4ddd-a0b5-2ab2ac20de61 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:38.199218 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424b2f7-0c5f-4ddd-a0b5-2ab2ac20de61 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:38.219155 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe1d97-5759-4077-8f99-6858d8b38fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:38.219866 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe1d97-5759-4077-8f99-6858d8b38fc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:38.220152 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe1d97-5759-4077-8f99-6858d8b38fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:38.220500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe1d97-5759-4077-8f99-6858d8b38fc5 admin admin] PATH_INFO: `/v3/projects/fa309bf373724f02b2aeabdd89f7c06a/users/8419f2de272d465584c95ccfac5b474e/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:38.230694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:15:38.255308 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fe1d97-5759-4077-8f99-6858d8b38fc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa309bf373724f02b2aeabdd89f7c06a, user_id=8419f2de272d465584c95ccfac5b474e, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:38.258491 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fe1d97-5759-4077-8f99-6858d8b38fc5 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:38.350707 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1191/2379] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:15:38 2022] PUT /identity/v3/projects/fa309bf373724f02b2aeabdd89f7c06a/users/8419f2de272d465584c95ccfac5b474e/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:15:38.361318 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8075ae-83ff-447a-96ea-e79de4c153d6 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:38.361655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8075ae-83ff-447a-96ea-e79de4c153d6 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:38.361885 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8075ae-83ff-447a-96ea-e79de4c153d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:38.424961 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-2a8075ae-83ff-447a-96ea-e79de4c153d6 None None] MFA Rules not processed for user `8419f2de272d465584c95ccfac5b474e`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:38.426419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2a8075ae-83ff-447a-96ea-e79de4c153d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:38.614794 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1189/2380] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:15:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:15:38.616286 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe1d97-5759-4077-8f99-6858d8b38fc5 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:38.822538 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe1d97-5759-4077-8f99-6858d8b38fc5 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:38.830509 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84123840-b538-4098-aa9c-5cfad9037a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:38.831572 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84123840-b538-4098-aa9c-5cfad9037a37 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:38.831858 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84123840-b538-4098-aa9c-5cfad9037a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:38.832140 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84123840-b538-4098-aa9c-5cfad9037a37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:38.832834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84123840-b538-4098-aa9c-5cfad9037a37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:38.834907 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84123840-b538-4098-aa9c-5cfad9037a37 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:39.471385 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1192/2381] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:15:38 2022] POST /identity/v3/projects => generated 375 bytes in 856 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:15:39.475226 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8075ae-83ff-447a-96ea-e79de4c153d6 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:39.491922 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8075ae-83ff-447a-96ea-e79de4c153d6 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:39.572108 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8faac-825d-4081-a96a-06fb8e861f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:39.573052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8faac-825d-4081-a96a-06fb8e861f5b admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:39.573439 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8faac-825d-4081-a96a-06fb8e861f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:39.573792 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8faac-825d-4081-a96a-06fb8e861f5b admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:39.575305 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e8faac-825d-4081-a96a-06fb8e861f5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:39.578904 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e8faac-825d-4081-a96a-06fb8e861f5b admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:39.676962 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1190/2382] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:15:39 2022] POST /identity/v3/users => generated 288 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:15:39.681111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84123840-b538-4098-aa9c-5cfad9037a37 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:39.701409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84123840-b538-4098-aa9c-5cfad9037a37 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:39.721700 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70230f-6971-4c15-a62e-010bd8f6643a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:39.722500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70230f-6971-4c15-a62e-010bd8f6643a admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:39.722787 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70230f-6971-4c15-a62e-010bd8f6643a admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:39.723623 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70230f-6971-4c15-a62e-010bd8f6643a admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:39.724343 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b70230f-6971-4c15-a62e-010bd8f6643a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:39.729128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b70230f-6971-4c15-a62e-010bd8f6643a admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:39.760056 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8faac-825d-4081-a96a-06fb8e861f5b admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:39.764473 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1193/2383] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:15:39.781694 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8faac-825d-4081-a96a-06fb8e861f5b admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:39.799544 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8899a9-d3f1-41dd-bc3d-7a3b4b9adacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:39.800493 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8899a9-d3f1-41dd-bc3d-7a3b4b9adacf admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:39.800786 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8899a9-d3f1-41dd-bc3d-7a3b4b9adacf admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:39.801066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8899a9-d3f1-41dd-bc3d-7a3b4b9adacf admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:39.801793 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8899a9-d3f1-41dd-bc3d-7a3b4b9adacf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:39.803566 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8899a9-d3f1-41dd-bc3d-7a3b4b9adacf admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:39.840758 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1191/2384] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:15:39.845478 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70230f-6971-4c15-a62e-010bd8f6643a admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:39.857677 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70230f-6971-4c15-a62e-010bd8f6643a admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:39.870498 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b663c-7028-4f87-867b-8aaad1acb431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:39.871613 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b663c-7028-4f87-867b-8aaad1acb431 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:39.872115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b663c-7028-4f87-867b-8aaad1acb431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:39.872754 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b663c-7028-4f87-867b-8aaad1acb431 admin admin] PATH_INFO: `/v3/projects/0828253286244d5a8042a2551616189c/users/d32055d1f81a4d859e384af9b05340f1/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:39.893481 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b663c-7028-4f87-867b-8aaad1acb431 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0828253286244d5a8042a2551616189c, user_id=d32055d1f81a4d859e384af9b05340f1, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:39.897027 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b663c-7028-4f87-867b-8aaad1acb431 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:39.974983 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1194/2385] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:15:39 2022] PUT /identity/v3/projects/0828253286244d5a8042a2551616189c/users/d32055d1f81a4d859e384af9b05340f1/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:15:39.990442 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac520c1-3dcc-4839-ac24-103677866dd0 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:39.991456 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac520c1-3dcc-4839-ac24-103677866dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:39.993719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac520c1-3dcc-4839-ac24-103677866dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:40.052652 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-dac520c1-3dcc-4839-ac24-103677866dd0 None None] MFA Rules not processed for user `d32055d1f81a4d859e384af9b05340f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:40.053960 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-dac520c1-3dcc-4839-ac24-103677866dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:40.098420 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:15:40.226940 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1192/2386] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:15:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:15:40.235193 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b663c-7028-4f87-867b-8aaad1acb431 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:40.318602 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b663c-7028-4f87-867b-8aaad1acb431 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:40.323167 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fc143-e8bc-48b8-9561-be7c754576e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:40.323781 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fc143-e8bc-48b8-9561-be7c754576e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:40.324062 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fc143-e8bc-48b8-9561-be7c754576e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:40.324369 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fc143-e8bc-48b8-9561-be7c754576e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:40.325033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0fc143-e8bc-48b8-9561-be7c754576e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:40.327002 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0fc143-e8bc-48b8-9561-be7c754576e2 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:40.473603 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1195/2387] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:15:40 2022] POST /identity/v3/projects => generated 377 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:15:40.477611 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac520c1-3dcc-4839-ac24-103677866dd0 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:40.489720 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac520c1-3dcc-4839-ac24-103677866dd0 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:40.592836 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea4f73-2466-4e8f-8024-68bb7e909413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:40.593486 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea4f73-2466-4e8f-8024-68bb7e909413 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:40.593770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea4f73-2466-4e8f-8024-68bb7e909413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:40.594235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea4f73-2466-4e8f-8024-68bb7e909413 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:40.594955 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea4f73-2466-4e8f-8024-68bb7e909413 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:40.599412 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea4f73-2466-4e8f-8024-68bb7e909413 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:40.706969 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1193/2388] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:15:40 2022] POST /identity/v3/users => generated 295 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:15:40.711316 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fc143-e8bc-48b8-9561-be7c754576e2 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:40.724283 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fc143-e8bc-48b8-9561-be7c754576e2 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:40.729784 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e27177-d260-454f-9702-f589d639df03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:40.730358 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e27177-d260-454f-9702-f589d639df03 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:40.730674 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e27177-d260-454f-9702-f589d639df03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:40.730991 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e27177-d260-454f-9702-f589d639df03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:40.731857 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e27177-d260-454f-9702-f589d639df03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:40.733524 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e27177-d260-454f-9702-f589d639df03 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:40.745814 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1196/2389] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:15:40.750467 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea4f73-2466-4e8f-8024-68bb7e909413 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:40.765199 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea4f73-2466-4e8f-8024-68bb7e909413 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:40.786179 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ef783-60d5-4eab-9c5e-d5604abf09b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:40.787117 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ef783-60d5-4eab-9c5e-d5604abf09b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:40.787265 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ef783-60d5-4eab-9c5e-d5604abf09b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:40.787515 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ef783-60d5-4eab-9c5e-d5604abf09b8 admin admin] PATH_INFO: `/v3/projects/43920308773b4bf4bd44579bc9be4dd0/users/087d8a335c064fe1b63b1d9e3058d97c/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:40.831360 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0ef783-60d5-4eab-9c5e-d5604abf09b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43920308773b4bf4bd44579bc9be4dd0, user_id=087d8a335c064fe1b63b1d9e3058d97c, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:40.844007 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0ef783-60d5-4eab-9c5e-d5604abf09b8 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:40.944314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1194/2390] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:15:40 2022] PUT /identity/v3/projects/43920308773b4bf4bd44579bc9be4dd0/users/087d8a335c064fe1b63b1d9e3058d97c/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:15:40.950353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e27177-d260-454f-9702-f589d639df03 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:40.983204 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf3e0c-084a-42c6-9c4c-52adf22f3865 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:40.983440 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf3e0c-084a-42c6-9c4c-52adf22f3865 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:40.983863 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf3e0c-084a-42c6-9c4c-52adf22f3865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:40.990901 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e27177-d260-454f-9702-f589d639df03 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:41.002449 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:15:41.006919 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-9ccf3e0c-084a-42c6-9c4c-52adf22f3865 None None] MFA Rules not processed for user `79539af2999d48baa11734ced32dccb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:41.008155 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9ccf3e0c-084a-42c6-9c4c-52adf22f3865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:41.095441 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1195/2391] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:15:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:15:41.156740 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336d722-6fa6-40e4-b983-c6a8cfb0fd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:41.157355 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336d722-6fa6-40e4-b983-c6a8cfb0fd07 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:41.157631 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336d722-6fa6-40e4-b983-c6a8cfb0fd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:41.157908 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336d722-6fa6-40e4-b983-c6a8cfb0fd07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:41.158665 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336d722-6fa6-40e4-b983-c6a8cfb0fd07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:41.160751 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336d722-6fa6-40e4-b983-c6a8cfb0fd07 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:41.173380 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1197/2392] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:15:41.178347 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf3e0c-084a-42c6-9c4c-52adf22f3865 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:41.195323 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf3e0c-084a-42c6-9c4c-52adf22f3865 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:41.201359 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0226ff-c123-4b0e-a3b7-cab1514e5e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:41.202130 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0226ff-c123-4b0e-a3b7-cab1514e5e5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:41.202979 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0226ff-c123-4b0e-a3b7-cab1514e5e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:41.203320 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0226ff-c123-4b0e-a3b7-cab1514e5e5a admin admin] PATH_INFO: `/v3/projects/43920308773b4bf4bd44579bc9be4dd0/users/087d8a335c064fe1b63b1d9e3058d97c/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:41.205668 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0226ff-c123-4b0e-a3b7-cab1514e5e5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=43920308773b4bf4bd44579bc9be4dd0, user_id=087d8a335c064fe1b63b1d9e3058d97c, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:41.209035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0226ff-c123-4b0e-a3b7-cab1514e5e5a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:41.224529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1196/2393] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:15:41 2022] PUT /identity/v3/projects/43920308773b4bf4bd44579bc9be4dd0/users/087d8a335c064fe1b63b1d9e3058d97c/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:15:41.236141 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840363d-7be1-4fae-8c51-787de0286703 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:41.236437 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840363d-7be1-4fae-8c51-787de0286703 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:41.236793 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840363d-7be1-4fae-8c51-787de0286703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:41.274217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-1840363d-7be1-4fae-8c51-787de0286703 None None] MFA Rules not processed for user `087d8a335c064fe1b63b1d9e3058d97c`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:41.275992 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-1840363d-7be1-4fae-8c51-787de0286703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:41.413914 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1198/2394] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:15:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:15:41.435273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0226ff-c123-4b0e-a3b7-cab1514e5e5a admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:41.438190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3c0226ff-c123-4b0e-a3b7-cab1514e5e5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:41.485956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:15:41.557010 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0226ff-c123-4b0e-a3b7-cab1514e5e5a admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:41.561602 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a6b8f-590f-4311-96de-f13568ab02d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:41.562524 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a6b8f-590f-4311-96de-f13568ab02d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:41.563052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a6b8f-590f-4311-96de-f13568ab02d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:41.564294 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a6b8f-590f-4311-96de-f13568ab02d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:41.666940 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a6b8f-590f-4311-96de-f13568ab02d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:41.668499 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a6b8f-590f-4311-96de-f13568ab02d8 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:41.682546 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1197/2395] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:15:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:15:43.798780 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840363d-7be1-4fae-8c51-787de0286703 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:43.813337 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840363d-7be1-4fae-8c51-787de0286703 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:43.817587 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be427f-b433-4ea6-97db-f605b4c9ab72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:43.818467 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be427f-b433-4ea6-97db-f605b4c9ab72 service cinder] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:43.819045 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be427f-b433-4ea6-97db-f605b4c9ab72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:43.820163 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be427f-b433-4ea6-97db-f605b4c9ab72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:43.893855 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be427f-b433-4ea6-97db-f605b4c9ab72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:43.895464 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be427f-b433-4ea6-97db-f605b4c9ab72 service cinder] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:43.906765 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1199/2396] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:15:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:15:45.820277 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a6b8f-590f-4311-96de-f13568ab02d8 service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:45.832607 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a6b8f-590f-4311-96de-f13568ab02d8 service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:45.837035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6827bdf8-fa6f-47e5-8b50-ae370df96c7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:45.837678 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827bdf8-fa6f-47e5-8b50-ae370df96c7f service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:45.840217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827bdf8-fa6f-47e5-8b50-ae370df96c7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:45.840496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827bdf8-fa6f-47e5-8b50-ae370df96c7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:45.855813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6827bdf8-fa6f-47e5-8b50-ae370df96c7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:45.857847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6827bdf8-fa6f-47e5-8b50-ae370df96c7f service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:45.877625 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1198/2397] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:15:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:15:48.571470 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52151ebd-4315-4cb8-9a57-ba3edcb981cc None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:48.572667 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52151ebd-4315-4cb8-9a57-ba3edcb981cc None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:48.573143 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52151ebd-4315-4cb8-9a57-ba3edcb981cc None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:48.577896 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1200/2398] 10.222.0.40 () {54 vars in 921 bytes} [Wed Aug 3 05:15:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:15:48.584301 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fa74f-1522-4ac9-8070-7de131f83529 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:48.585198 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fa74f-1522-4ac9-8070-7de131f83529 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:48.585696 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fa74f-1522-4ac9-8070-7de131f83529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:48.614416 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-250fa74f-1522-4ac9-8070-7de131f83529 None None] MFA Rules not processed for user `a3fa4a6c16274c18849869b55b33ac62`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:48.616078 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-250fa74f-1522-4ac9-8070-7de131f83529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:48.748253 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1199/2399] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Aug 3 05:15:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:15:50.822294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52151ebd-4315-4cb8-9a57-ba3edcb981cc None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:50.855670 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:15:50.890108 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52151ebd-4315-4cb8-9a57-ba3edcb981cc None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:50.907693 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ed9ec-1bc7-4d2f-8046-d9070df211ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:50.907693 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ed9ec-1bc7-4d2f-8046-d9070df211ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:50.907693 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ed9ec-1bc7-4d2f-8046-d9070df211ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:50.907693 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ed9ec-1bc7-4d2f-8046-d9070df211ad admin admin] PATH_INFO: `/v3/users/8419f2de272d465584c95ccfac5b474e` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:50.908014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ed9ec-1bc7-4d2f-8046-d9070df211ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=8419f2de272d465584c95ccfac5b474e)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:50.908014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ed9ec-1bc7-4d2f-8046-d9070df211ad admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:51.170743 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-213ed9ec-1bc7-4d2f-8046-d9070df211ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8419f2de272d465584c95ccfac5b474e'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:51.270287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-213ed9ec-1bc7-4d2f-8046-d9070df211ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8419f2de272d465584c95ccfac5b474e'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:51.274095 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-213ed9ec-1bc7-4d2f-8046-d9070df211ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8419f2de272d465584c95ccfac5b474e'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:51.277952 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250fa74f-1522-4ac9-8070-7de131f83529 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:51.280067 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-250fa74f-1522-4ac9-8070-7de131f83529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:51.337818 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-213ed9ec-1bc7-4d2f-8046-d9070df211ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8419f2de272d465584c95ccfac5b474e'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:51.341389 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-213ed9ec-1bc7-4d2f-8046-d9070df211ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8419f2de272d465584c95ccfac5b474e'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:51.366657 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1201/2400] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:15:50 2022] DELETE /identity/v3/users/8419f2de272d465584c95ccfac5b474e => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:15:51.371774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e4c93-7c24-42dd-94ce-bf7dabcf8e4d None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:51.378521 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250fa74f-1522-4ac9-8070-7de131f83529 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:51.382207 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e4c93-7c24-42dd-94ce-bf7dabcf8e4d None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:51.382634 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e4c93-7c24-42dd-94ce-bf7dabcf8e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:51.384248 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:15:51.414663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-6f9e4c93-7c24-42dd-94ce-bf7dabcf8e4d None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:51.416046 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-6f9e4c93-7c24-42dd-94ce-bf7dabcf8e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:51.444905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d326164f-1a89-4885-ac03-fdcdcf3ce61f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cc14b33895443b893fab487bfe1858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:51.445853 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326164f-1a89-4885-ac03-fdcdcf3ce61f service glance] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:51.446315 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326164f-1a89-4885-ac03-fdcdcf3ce61f service glance] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:51.446773 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326164f-1a89-4885-ac03-fdcdcf3ce61f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:51.449859 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d326164f-1a89-4885-ac03-fdcdcf3ce61f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:51.512990 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1202/2401] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:15:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:15:51.522932 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e4c93-7c24-42dd-94ce-bf7dabcf8e4d None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:51.524485 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-6f9e4c93-7c24-42dd-94ce-bf7dabcf8e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:51.567959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d326164f-1a89-4885-ac03-fdcdcf3ce61f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:51.572023 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d326164f-1a89-4885-ac03-fdcdcf3ce61f service glance] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:51.609079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:15:51.615609 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1200/2402] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:15:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:15:51.618203 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e4c93-7c24-42dd-94ce-bf7dabcf8e4d None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:51.624952 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b423379-e956-492f-9cbe-afbda46ff214 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:51.627260 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b423379-e956-492f-9cbe-afbda46ff214 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:51.627870 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b423379-e956-492f-9cbe-afbda46ff214 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:51.628310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b423379-e956-492f-9cbe-afbda46ff214 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:51.639149 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b423379-e956-492f-9cbe-afbda46ff214 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:51.641036 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b423379-e956-492f-9cbe-afbda46ff214 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:51.658357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1203/2403] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:15:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:15:51.915958 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd38641-dfee-4dbc-becb-210fb4067cb1 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:51.916254 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd38641-dfee-4dbc-becb-210fb4067cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:51.916539 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd38641-dfee-4dbc-becb-210fb4067cb1 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:51.919056 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1201/2404] 10.222.0.40 () {54 vars in 921 bytes} [Wed Aug 3 05:15:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:15:51.928394 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10d0ec-4fe0-43cc-9879-e15e8349b538 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:51.929247 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10d0ec-4fe0-43cc-9879-e15e8349b538 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:51.929533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10d0ec-4fe0-43cc-9879-e15e8349b538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:51.981739 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-4e10d0ec-4fe0-43cc-9879-e15e8349b538 None None] MFA Rules not processed for user `cff15ea4cbc64b068aa5f6586b5bff87`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:51.981739 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-4e10d0ec-4fe0-43cc-9879-e15e8349b538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:51.984131 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1204/2405] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Aug 3 05:15:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:15:52.016394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd38641-dfee-4dbc-becb-210fb4067cb1 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:52.031112 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd38641-dfee-4dbc-becb-210fb4067cb1 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:52.035280 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51a655-0eb5-469e-8603-73f0c90eaf23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:52.036847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51a655-0eb5-469e-8603-73f0c90eaf23 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:52.037146 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51a655-0eb5-469e-8603-73f0c90eaf23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:52.037432 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51a655-0eb5-469e-8603-73f0c90eaf23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:52.059260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51a655-0eb5-469e-8603-73f0c90eaf23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:52.060868 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51a655-0eb5-469e-8603-73f0c90eaf23 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:52.074350 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1202/2406] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:15:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:15:52.231500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c9e68-3237-4756-ba74-71c8451c8537 None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:52.231867 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c9e68-3237-4756-ba74-71c8451c8537 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:52.232756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c9e68-3237-4756-ba74-71c8451c8537 None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:52.234954 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1205/2407] 10.222.0.40 () {54 vars in 921 bytes} [Wed Aug 3 05:15:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 05:15:52.242534 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c84c74-d92e-4773-91e8-3f8e2b495eac None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:52.242772 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c84c74-d92e-4773-91e8-3f8e2b495eac None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:52.243160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c84c74-d92e-4773-91e8-3f8e2b495eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:52.286617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-a7c84c74-d92e-4773-91e8-3f8e2b495eac None None] MFA Rules not processed for user `a3fa4a6c16274c18849869b55b33ac62`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:52.292175 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a7c84c74-d92e-4773-91e8-3f8e2b495eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:52.300965 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1203/2408] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Aug 3 05:15:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:15:52.329518 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c9e68-3237-4756-ba74-71c8451c8537 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:52.348697 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c9e68-3237-4756-ba74-71c8451c8537 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:52.353480 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a429d-13e2-433c-a1c5-17bfeeefde21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:52.354359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a429d-13e2-433c-a1c5-17bfeeefde21 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:52.354905 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a429d-13e2-433c-a1c5-17bfeeefde21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:52.355415 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a429d-13e2-433c-a1c5-17bfeeefde21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:52.388031 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a429d-13e2-433c-a1c5-17bfeeefde21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:52.389293 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a429d-13e2-433c-a1c5-17bfeeefde21 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:52.410884 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1206/2409] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:15:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:15:52.571273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c84c74-d92e-4773-91e8-3f8e2b495eac None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:52.573001 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a7c84c74-d92e-4773-91e8-3f8e2b495eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:52.585580 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf9581-c196-41da-befb-5a7591a9c708 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:52.585876 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf9581-c196-41da-befb-5a7591a9c708 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:52.586162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf9581-c196-41da-befb-5a7591a9c708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:52.612637 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c84c74-d92e-4773-91e8-3f8e2b495eac None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:52.635602 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3b5f1-92f5-4fe4-ba77-64cc2ca395d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:52.650318 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3b5f1-92f5-4fe4-ba77-64cc2ca395d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:52.650685 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3b5f1-92f5-4fe4-ba77-64cc2ca395d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:52.651451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-40cf9581-c196-41da-befb-5a7591a9c708 None None] MFA Rules not processed for user `de853a2dfffb4f05b6f47df2280ae4e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:52.653407 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-40cf9581-c196-41da-befb-5a7591a9c708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:52.654591 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3b5f1-92f5-4fe4-ba77-64cc2ca395d8 admin admin] PATH_INFO: `/v3/projects/fa309bf373724f02b2aeabdd89f7c06a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:52.657477 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec3b5f1-92f5-4fe4-ba77-64cc2ca395d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa309bf373724f02b2aeabdd89f7c06a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:52.674312 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec3b5f1-92f5-4fe4-ba77-64cc2ca395d8 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:52.791672 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1207/2410] 10.222.0.40 () {58 vars in 967 bytes} [Wed Aug 3 05:15:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:15:52.825983 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf9581-c196-41da-befb-5a7591a9c708 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:52.828571 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-40cf9581-c196-41da-befb-5a7591a9c708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:52.869820 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-1ec3b5f1-92f5-4fe4-ba77-64cc2ca395d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa309bf373724f02b2aeabdd89f7c06a', 'request_id': 'req-1ec3b5f1-92f5-4fe4-ba77-64cc2ca395d8'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:52.952855 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-1ec3b5f1-92f5-4fe4-ba77-64cc2ca395d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa309bf373724f02b2aeabdd89f7c06a', 'request_id': 'req-1ec3b5f1-92f5-4fe4-ba77-64cc2ca395d8'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:52.959374 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-1ec3b5f1-92f5-4fe4-ba77-64cc2ca395d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa309bf373724f02b2aeabdd89f7c06a', 'request_id': 'req-1ec3b5f1-92f5-4fe4-ba77-64cc2ca395d8'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:52.965225 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf9581-c196-41da-befb-5a7591a9c708 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:52.974279 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-1ec3b5f1-92f5-4fe4-ba77-64cc2ca395d8 admin admin] The token cache is being invalidate because project fa309bf373724f02b2aeabdd89f7c06a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:15:52.974648 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-1ec3b5f1-92f5-4fe4-ba77-64cc2ca395d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:52.978176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-1ec3b5f1-92f5-4fe4-ba77-64cc2ca395d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:52.978176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1204/2411] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:15:52 2022] DELETE /identity/v3/projects/fa309bf373724f02b2aeabdd89f7c06a => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:15:52.983104 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3b5f1-92f5-4fe4-ba77-64cc2ca395d8 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:52.985745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1ec3b5f1-92f5-4fe4-ba77-64cc2ca395d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:53.008629 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:15:53.062743 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a9e10-259b-4909-85c0-6c4be06b053b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1014bcb4d36441aa46042941c9a63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:53.063480 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a9e10-259b-4909-85c0-6c4be06b053b service swift] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:53.063768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a9e10-259b-4909-85c0-6c4be06b053b service swift] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:53.064046 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a9e10-259b-4909-85c0-6c4be06b053b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:53.066029 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-246a9e10-259b-4909-85c0-6c4be06b053b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:53.070192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3b5f1-92f5-4fe4-ba77-64cc2ca395d8 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:53.075221 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada6b8c-1438-4725-a16c-8b9de239c1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:53.075859 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada6b8c-1438-4725-a16c-8b9de239c1aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:53.076143 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada6b8c-1438-4725-a16c-8b9de239c1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:53.076425 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada6b8c-1438-4725-a16c-8b9de239c1aa admin admin] PATH_INFO: `/v3/users/d32055d1f81a4d859e384af9b05340f1` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:53.077640 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada6b8c-1438-4725-a16c-8b9de239c1aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=d32055d1f81a4d859e384af9b05340f1)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:53.079615 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada6b8c-1438-4725-a16c-8b9de239c1aa admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:53.085089 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:15:53.136855 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246a9e10-259b-4909-85c0-6c4be06b053b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:53.138583 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246a9e10-259b-4909-85c0-6c4be06b053b service swift] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:53.147077 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1208/2412] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 3 05:15:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 05:15:53.333719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9ada6b8c-1438-4725-a16c-8b9de239c1aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd32055d1f81a4d859e384af9b05340f1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:53.422391 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9ada6b8c-1438-4725-a16c-8b9de239c1aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd32055d1f81a4d859e384af9b05340f1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:53.423533 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9ada6b8c-1438-4725-a16c-8b9de239c1aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd32055d1f81a4d859e384af9b05340f1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:53.439785 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9ada6b8c-1438-4725-a16c-8b9de239c1aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd32055d1f81a4d859e384af9b05340f1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:53.440886 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9ada6b8c-1438-4725-a16c-8b9de239c1aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd32055d1f81a4d859e384af9b05340f1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:53.456374 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1205/2413] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:15:52 2022] DELETE /identity/v3/users/d32055d1f81a4d859e384af9b05340f1 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:15:54.095591 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a9e10-259b-4909-85c0-6c4be06b053b service swift] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:54.113060 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a9e10-259b-4909-85c0-6c4be06b053b service swift] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:54.182349 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c163f-7d30-4bdd-a305-9fcad42402a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:54.182994 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c163f-7d30-4bdd-a305-9fcad42402a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:54.183292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c163f-7d30-4bdd-a305-9fcad42402a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:54.183566 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c163f-7d30-4bdd-a305-9fcad42402a0 admin admin] PATH_INFO: `/v3/projects/0828253286244d5a8042a2551616189c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:54.184745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c163f-7d30-4bdd-a305-9fcad42402a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0828253286244d5a8042a2551616189c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:54.186678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c163f-7d30-4bdd-a305-9fcad42402a0 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:54.316839 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-4f2c163f-7d30-4bdd-a305-9fcad42402a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0828253286244d5a8042a2551616189c', 'request_id': 'req-4f2c163f-7d30-4bdd-a305-9fcad42402a0'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:54.325077 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-4f2c163f-7d30-4bdd-a305-9fcad42402a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0828253286244d5a8042a2551616189c', 'request_id': 'req-4f2c163f-7d30-4bdd-a305-9fcad42402a0'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:54.427201 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-4f2c163f-7d30-4bdd-a305-9fcad42402a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0828253286244d5a8042a2551616189c', 'request_id': 'req-4f2c163f-7d30-4bdd-a305-9fcad42402a0'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:54.442956 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-4f2c163f-7d30-4bdd-a305-9fcad42402a0 admin admin] The token cache is being invalidate because project 0828253286244d5a8042a2551616189c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:15:54.443255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-4f2c163f-7d30-4bdd-a305-9fcad42402a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:54.443926 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-4f2c163f-7d30-4bdd-a305-9fcad42402a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:54.445514 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1209/2414] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:15:54 2022] DELETE /identity/v3/projects/0828253286244d5a8042a2551616189c => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:15:54.450360 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada6b8c-1438-4725-a16c-8b9de239c1aa admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:54.452018 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9ada6b8c-1438-4725-a16c-8b9de239c1aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:54.465751 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada6b8c-1438-4725-a16c-8b9de239c1aa admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:54.528411 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800594bf-91b9-4936-88f8-beb5b22cfda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:54.529098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800594bf-91b9-4936-88f8-beb5b22cfda2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:54.529421 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800594bf-91b9-4936-88f8-beb5b22cfda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:54.529674 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800594bf-91b9-4936-88f8-beb5b22cfda2 admin admin] PATH_INFO: `/v3/users/087d8a335c064fe1b63b1d9e3058d97c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:54.530906 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800594bf-91b9-4936-88f8-beb5b22cfda2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=087d8a335c064fe1b63b1d9e3058d97c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:54.532843 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800594bf-91b9-4936-88f8-beb5b22cfda2 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:54.712788 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:15:54.727435 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-800594bf-91b9-4936-88f8-beb5b22cfda2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '087d8a335c064fe1b63b1d9e3058d97c'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:54.734996 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c163f-7d30-4bdd-a305-9fcad42402a0 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:54.738466 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-4f2c163f-7d30-4bdd-a305-9fcad42402a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:54.790476 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:15:54.815484 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c163f-7d30-4bdd-a305-9fcad42402a0 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:54.818944 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-800594bf-91b9-4936-88f8-beb5b22cfda2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '087d8a335c064fe1b63b1d9e3058d97c'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:54.819568 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-800594bf-91b9-4936-88f8-beb5b22cfda2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '087d8a335c064fe1b63b1d9e3058d97c'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:54.820661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d0dd6f-f10e-4cab-8d45-e2ed845b687b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59846d88f4314c2382aa5cb8bae5053b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:54.821289 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0dd6f-f10e-4cab-8d45-e2ed845b687b service placement] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:54.821577 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0dd6f-f10e-4cab-8d45-e2ed845b687b service placement] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:54.821857 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0dd6f-f10e-4cab-8d45-e2ed845b687b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:54.823749 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-69d0dd6f-f10e-4cab-8d45-e2ed845b687b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:54.832505 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-800594bf-91b9-4936-88f8-beb5b22cfda2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '087d8a335c064fe1b63b1d9e3058d97c'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:54.833145 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-800594bf-91b9-4936-88f8-beb5b22cfda2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '087d8a335c064fe1b63b1d9e3058d97c'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:54.835612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d0dd6f-f10e-4cab-8d45-e2ed845b687b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:54.837269 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d0dd6f-f10e-4cab-8d45-e2ed845b687b service placement] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:54.846002 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1206/2415] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:15:54 2022] DELETE /identity/v3/users/087d8a335c064fe1b63b1d9e3058d97c => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:15:54.848011 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1210/2416] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:15:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:15:55.349790 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800594bf-91b9-4936-88f8-beb5b22cfda2 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:55.351718 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-800594bf-91b9-4936-88f8-beb5b22cfda2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:55.422383 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800594bf-91b9-4936-88f8-beb5b22cfda2 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:55.427059 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a013c-df81-4130-b12c-3f9fd1311bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:55.427713 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a013c-df81-4130-b12c-3f9fd1311bfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:55.428004 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a013c-df81-4130-b12c-3f9fd1311bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:55.428285 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a013c-df81-4130-b12c-3f9fd1311bfa admin admin] PATH_INFO: `/v3/projects/43920308773b4bf4bd44579bc9be4dd0` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:55.429447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a013c-df81-4130-b12c-3f9fd1311bfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=43920308773b4bf4bd44579bc9be4dd0)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:55.431488 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a013c-df81-4130-b12c-3f9fd1311bfa admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:55.539875 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-677a013c-df81-4130-b12c-3f9fd1311bfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43920308773b4bf4bd44579bc9be4dd0', 'request_id': 'req-677a013c-df81-4130-b12c-3f9fd1311bfa'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:55.610189 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-677a013c-df81-4130-b12c-3f9fd1311bfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43920308773b4bf4bd44579bc9be4dd0', 'request_id': 'req-677a013c-df81-4130-b12c-3f9fd1311bfa'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:55.618055 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-677a013c-df81-4130-b12c-3f9fd1311bfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43920308773b4bf4bd44579bc9be4dd0', 'request_id': 'req-677a013c-df81-4130-b12c-3f9fd1311bfa'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:55.632209 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-677a013c-df81-4130-b12c-3f9fd1311bfa admin admin] The token cache is being invalidate because project 43920308773b4bf4bd44579bc9be4dd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:15:55.632500 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-677a013c-df81-4130-b12c-3f9fd1311bfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:55.633149 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-677a013c-df81-4130-b12c-3f9fd1311bfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:55.634691 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1207/2417] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:15:55 2022] DELETE /identity/v3/projects/43920308773b4bf4bd44579bc9be4dd0 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:15:55.640382 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb8bd8-550a-4503-8b07-af2687292250 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:55.640889 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb8bd8-550a-4503-8b07-af2687292250 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:55.640998 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb8bd8-550a-4503-8b07-af2687292250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:55.661818 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-abbb8bd8-550a-4503-8b07-af2687292250 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:55.663159 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-abbb8bd8-550a-4503-8b07-af2687292250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:55.728457 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1211/2418] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:15:55.737637 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716664ae-8378-41a6-bfa0-739524d421d6 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:55.737956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716664ae-8378-41a6-bfa0-739524d421d6 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:55.738274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716664ae-8378-41a6-bfa0-739524d421d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:55.758288 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-716664ae-8378-41a6-bfa0-739524d421d6 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:55.759556 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-716664ae-8378-41a6-bfa0-739524d421d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:55.768089 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1208/2419] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:15:55.773837 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb8bd8-550a-4503-8b07-af2687292250 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:55.783502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb8bd8-550a-4503-8b07-af2687292250 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:55.788252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e5b5c-a7d4-4722-a6eb-e3b6dbf29bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:55.788929 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e5b5c-a7d4-4722-a6eb-e3b6dbf29bbf admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:55.789217 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e5b5c-a7d4-4722-a6eb-e3b6dbf29bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:55.789498 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e5b5c-a7d4-4722-a6eb-e3b6dbf29bbf admin admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:55.790203 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e5b5c-a7d4-4722-a6eb-e3b6dbf29bbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:15:55.790513 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e5b5c-a7d4-4722-a6eb-e3b6dbf29bbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:55.792189 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e5b5c-a7d4-4722-a6eb-e3b6dbf29bbf admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:55.805255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1212/2420] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:15:55.809860 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716664ae-8378-41a6-bfa0-739524d421d6 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:55.819634 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716664ae-8378-41a6-bfa0-739524d421d6 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:55.824927 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6a0c2-4ec2-49d3-959a-3904be3af7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:55.825659 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6a0c2-4ec2-49d3-959a-3904be3af7a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:55.825959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6a0c2-4ec2-49d3-959a-3904be3af7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:55.826349 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6a0c2-4ec2-49d3-959a-3904be3af7a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:55.827295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f6a0c2-4ec2-49d3-959a-3904be3af7a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:55.831182 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f6a0c2-4ec2-49d3-959a-3904be3af7a8 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:55.911430 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1209/2421] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:15:55 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:15:55.915674 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e5b5c-a7d4-4722-a6eb-e3b6dbf29bbf admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:55.926657 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e5b5c-a7d4-4722-a6eb-e3b6dbf29bbf admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:56.004825 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474175cd-c8ff-4a69-acf2-ab0dd3cad175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:56.005457 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474175cd-c8ff-4a69-acf2-ab0dd3cad175 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:56.005748 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474175cd-c8ff-4a69-acf2-ab0dd3cad175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:56.006030 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474175cd-c8ff-4a69-acf2-ab0dd3cad175 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:56.006863 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474175cd-c8ff-4a69-acf2-ab0dd3cad175 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:56.008842 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474175cd-c8ff-4a69-acf2-ab0dd3cad175 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:56.125687 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1213/2422] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:15:55 2022] POST /identity/v3/users => generated 288 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:15:56.130527 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6a0c2-4ec2-49d3-959a-3904be3af7a8 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:56.142889 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6a0c2-4ec2-49d3-959a-3904be3af7a8 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:56.162045 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f348de0f-4a42-4ef6-b7d9-6852b14a40f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:56.162972 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348de0f-4a42-4ef6-b7d9-6852b14a40f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:56.163265 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348de0f-4a42-4ef6-b7d9-6852b14a40f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:56.163546 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348de0f-4a42-4ef6-b7d9-6852b14a40f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:56.164319 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f348de0f-4a42-4ef6-b7d9-6852b14a40f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:56.166038 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f348de0f-4a42-4ef6-b7d9-6852b14a40f7 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:56.173674 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:15:56.184135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1210/2423] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:15:56.195027 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474175cd-c8ff-4a69-acf2-ab0dd3cad175 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:56.206638 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474175cd-c8ff-4a69-acf2-ab0dd3cad175 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:56.211523 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf78938-c4d8-4032-ac12-b0c47fbcd57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:56.212189 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf78938-c4d8-4032-ac12-b0c47fbcd57e admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:56.212474 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf78938-c4d8-4032-ac12-b0c47fbcd57e admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:56.212752 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf78938-c4d8-4032-ac12-b0c47fbcd57e admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:56.213459 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf78938-c4d8-4032-ac12-b0c47fbcd57e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:56.215439 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf78938-c4d8-4032-ac12-b0c47fbcd57e admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:56.228933 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1214/2424] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:15:56.233001 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f348de0f-4a42-4ef6-b7d9-6852b14a40f7 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:56.244748 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f348de0f-4a42-4ef6-b7d9-6852b14a40f7 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:56.266818 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4459e-626d-4913-a578-ecf7138f3b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:56.267566 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4459e-626d-4913-a578-ecf7138f3b52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:56.267856 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4459e-626d-4913-a578-ecf7138f3b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:56.268140 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4459e-626d-4913-a578-ecf7138f3b52 admin admin] PATH_INFO: `/v3/projects/7a970d4cbb3545a187c75848e10cb166/users/ff504c0570b24f3baa8744a9740bcfce/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:56.290032 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4459e-626d-4913-a578-ecf7138f3b52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a970d4cbb3545a187c75848e10cb166, user_id=ff504c0570b24f3baa8744a9740bcfce, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:56.293370 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4459e-626d-4913-a578-ecf7138f3b52 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:56.379186 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1211/2425] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:15:56 2022] PUT /identity/v3/projects/7a970d4cbb3545a187c75848e10cb166/users/ff504c0570b24f3baa8744a9740bcfce/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:15:56.391656 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2252067-ee12-4ff4-8338-5f2cfcae8f17 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:56.392353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2252067-ee12-4ff4-8338-5f2cfcae8f17 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:56.392449 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2252067-ee12-4ff4-8338-5f2cfcae8f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:56.444782 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-d2252067-ee12-4ff4-8338-5f2cfcae8f17 None None] MFA Rules not processed for user `ff504c0570b24f3baa8744a9740bcfce`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:56.446590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-d2252067-ee12-4ff4-8338-5f2cfcae8f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:56.490698 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:15:56.530060 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1215/2426] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:15:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:15:56.543999 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4459e-626d-4913-a578-ecf7138f3b52 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:56.547178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2da4459e-626d-4913-a578-ecf7138f3b52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:56.560399 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37753550-94ec-440c-b9b4-fcfc47f7eba6 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:56.560698 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37753550-94ec-440c-b9b4-fcfc47f7eba6 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:56.560935 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37753550-94ec-440c-b9b4-fcfc47f7eba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:56.583357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-37753550-94ec-440c-b9b4-fcfc47f7eba6 None None] MFA Rules not processed for user `de853a2dfffb4f05b6f47df2280ae4e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:56.588262 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-37753550-94ec-440c-b9b4-fcfc47f7eba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:56.635333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4459e-626d-4913-a578-ecf7138f3b52 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:56.641451 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac4882-e22e-4680-8db7-78b63add8351 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:56.642379 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac4882-e22e-4680-8db7-78b63add8351 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:56.643418 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac4882-e22e-4680-8db7-78b63add8351 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:56.643707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac4882-e22e-4680-8db7-78b63add8351 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:56.656305 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac4882-e22e-4680-8db7-78b63add8351 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:56.658124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac4882-e22e-4680-8db7-78b63add8351 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:56.672436 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1216/2427] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Aug 3 05:15:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:15:56.679019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1212/2428] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:15:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:15:56.689046 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37753550-94ec-440c-b9b4-fcfc47f7eba6 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:56.691945 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-37753550-94ec-440c-b9b4-fcfc47f7eba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:56.715537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a47636-1d3a-4da9-8b1a-4ad9a6728315 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:56.715854 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a47636-1d3a-4da9-8b1a-4ad9a6728315 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:56.716104 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a47636-1d3a-4da9-8b1a-4ad9a6728315 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:56.719099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1213/2429] 10.222.0.40 () {54 vars in 921 bytes} [Wed Aug 3 05:15:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 05:15:56.729276 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd362157-8cf1-4334-87ec-e96754c8e383 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:56.730469 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd362157-8cf1-4334-87ec-e96754c8e383 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:56.730858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd362157-8cf1-4334-87ec-e96754c8e383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:56.771819 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-dd362157-8cf1-4334-87ec-e96754c8e383 None None] MFA Rules not processed for user `59846d88f4314c2382aa5cb8bae5053b`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:56.774494 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-dd362157-8cf1-4334-87ec-e96754c8e383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:56.869360 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37753550-94ec-440c-b9b4-fcfc47f7eba6 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:56.884618 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0293d3-5fd8-4b34-bcbe-f3dbd463e372 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1014bcb4d36441aa46042941c9a63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:56.886111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0293d3-5fd8-4b34-bcbe-f3dbd463e372 service swift] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:56.886919 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0293d3-5fd8-4b34-bcbe-f3dbd463e372 service swift] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:56.887650 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0293d3-5fd8-4b34-bcbe-f3dbd463e372 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:56.906136 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0293d3-5fd8-4b34-bcbe-f3dbd463e372 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:56.908494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0293d3-5fd8-4b34-bcbe-f3dbd463e372 service swift] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:56.919850 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1217/2430] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 3 05:15:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 05:15:56.927031 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1214/2431] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Aug 3 05:15:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:15:56.937525 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0293d3-5fd8-4b34-bcbe-f3dbd463e372 service swift] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:56.945353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-dc0293d3-5fd8-4b34-bcbe-f3dbd463e372 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:56.953637 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd362157-8cf1-4334-87ec-e96754c8e383 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:56.955501 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-dd362157-8cf1-4334-87ec-e96754c8e383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:57.016191 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd362157-8cf1-4334-87ec-e96754c8e383 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:57.023224 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29612fd-0aa9-4cef-b692-de8c1eaa3db3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59846d88f4314c2382aa5cb8bae5053b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:57.024678 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29612fd-0aa9-4cef-b692-de8c1eaa3db3 service placement] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:57.025180 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29612fd-0aa9-4cef-b692-de8c1eaa3db3 service placement] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:57.025975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29612fd-0aa9-4cef-b692-de8c1eaa3db3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:57.028234 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0293d3-5fd8-4b34-bcbe-f3dbd463e372 service swift] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:57.035351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828efe4-d8bd-4300-87f6-4159be6bf7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:57.036856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828efe4-d8bd-4300-87f6-4159be6bf7b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:57.036856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828efe4-d8bd-4300-87f6-4159be6bf7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:57.037176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828efe4-d8bd-4300-87f6-4159be6bf7b6 admin admin] PATH_INFO: `/v3/users/8cbb45074cef4d5082580c4c5130eb9a` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:57.038350 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29612fd-0aa9-4cef-b692-de8c1eaa3db3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:57.038571 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828efe4-d8bd-4300-87f6-4159be6bf7b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cbb45074cef4d5082580c4c5130eb9a)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:57.040419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29612fd-0aa9-4cef-b692-de8c1eaa3db3 service placement] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:57.040562 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828efe4-d8bd-4300-87f6-4159be6bf7b6 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:57.053115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:15:57.056150 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1215/2432] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:15:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:15:57.264692 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-1828efe4-d8bd-4300-87f6-4159be6bf7b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cbb45074cef4d5082580c4c5130eb9a'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:57.360553 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-1828efe4-d8bd-4300-87f6-4159be6bf7b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cbb45074cef4d5082580c4c5130eb9a'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:57.362668 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-1828efe4-d8bd-4300-87f6-4159be6bf7b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cbb45074cef4d5082580c4c5130eb9a'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:57.412214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-1828efe4-d8bd-4300-87f6-4159be6bf7b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cbb45074cef4d5082580c4c5130eb9a'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:57.413010 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-1828efe4-d8bd-4300-87f6-4159be6bf7b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cbb45074cef4d5082580c4c5130eb9a'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:57.434525 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1218/2433] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:15:56 2022] DELETE /identity/v3/users/8cbb45074cef4d5082580c4c5130eb9a => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:15:57.731417 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f4d08-d92f-478d-a451-942acd2dcc8a None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:57.732200 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f4d08-d92f-478d-a451-942acd2dcc8a None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:57.732815 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f4d08-d92f-478d-a451-942acd2dcc8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:57.764579 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-2d6f4d08-d92f-478d-a451-942acd2dcc8a None None] MFA Rules not processed for user `de853a2dfffb4f05b6f47df2280ae4e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:57.772141 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2d6f4d08-d92f-478d-a451-942acd2dcc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:57.861855 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1216/2434] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Aug 3 05:15:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:15:57.888896 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828efe4-d8bd-4300-87f6-4159be6bf7b6 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:57.890523 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-1828efe4-d8bd-4300-87f6-4159be6bf7b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:58.044064 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828efe4-d8bd-4300-87f6-4159be6bf7b6 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:58.053389 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3ff29-b2a9-4264-a12d-e3d1ed11430b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1014bcb4d36441aa46042941c9a63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:58.054392 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3ff29-b2a9-4264-a12d-e3d1ed11430b service swift] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:58.055018 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3ff29-b2a9-4264-a12d-e3d1ed11430b service swift] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:58.055472 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3ff29-b2a9-4264-a12d-e3d1ed11430b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:58.071854 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3ff29-b2a9-4264-a12d-e3d1ed11430b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:58.074661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3ff29-b2a9-4264-a12d-e3d1ed11430b service swift] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:58.087334 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1219/2435] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 3 05:15:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 05:15:58.383794 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f4d08-d92f-478d-a451-942acd2dcc8a None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:58.386179 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2d6f4d08-d92f-478d-a451-942acd2dcc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:58.401882 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:15:58.512930 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f4d08-d92f-478d-a451-942acd2dcc8a None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:58.531735 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fcbb1f-5539-4c3a-b349-312134542946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:58.532769 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcbb1f-5539-4c3a-b349-312134542946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:58.533322 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcbb1f-5539-4c3a-b349-312134542946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:58.533800 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcbb1f-5539-4c3a-b349-312134542946 admin admin] PATH_INFO: `/v3/projects/d9b28cfcd2da41528eb3d71199d3b2d9` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:58.535786 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fcbb1f-5539-4c3a-b349-312134542946 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9b28cfcd2da41528eb3d71199d3b2d9)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:58.538154 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fcbb1f-5539-4c3a-b349-312134542946 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:58.754340 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906843f-1402-49f7-8eb3-1ac560f56c0f None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:58.754665 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906843f-1402-49f7-8eb3-1ac560f56c0f None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:58.754947 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906843f-1402-49f7-8eb3-1ac560f56c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:58.767708 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-99fcbb1f-5539-4c3a-b349-312134542946 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9b28cfcd2da41528eb3d71199d3b2d9', 'request_id': 'req-99fcbb1f-5539-4c3a-b349-312134542946'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:58.784696 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-8906843f-1402-49f7-8eb3-1ac560f56c0f None None] MFA Rules not processed for user `ff504c0570b24f3baa8744a9740bcfce`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:58.785911 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-8906843f-1402-49f7-8eb3-1ac560f56c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:58.952257 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1220/2436] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:15:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:15:59.375442 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-99fcbb1f-5539-4c3a-b349-312134542946 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9b28cfcd2da41528eb3d71199d3b2d9', 'request_id': 'req-99fcbb1f-5539-4c3a-b349-312134542946'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:59.391240 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-99fcbb1f-5539-4c3a-b349-312134542946 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9b28cfcd2da41528eb3d71199d3b2d9', 'request_id': 'req-99fcbb1f-5539-4c3a-b349-312134542946'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:59.405084 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-99fcbb1f-5539-4c3a-b349-312134542946 admin admin] The token cache is being invalidate because project d9b28cfcd2da41528eb3d71199d3b2d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:15:59.405388 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-99fcbb1f-5539-4c3a-b349-312134542946 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:59.406046 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-99fcbb1f-5539-4c3a-b349-312134542946 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:15:59.409293 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1217/2437] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:15:58 2022] DELETE /identity/v3/projects/d9b28cfcd2da41528eb3d71199d3b2d9 => generated 0 bytes in 1025 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:15:59.412862 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d7a4e-ccad-4f40-938a-5500040ab718 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:59.413155 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d7a4e-ccad-4f40-938a-5500040ab718 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:59.413437 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d7a4e-ccad-4f40-938a-5500040ab718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:59.435982 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-817d7a4e-ccad-4f40-938a-5500040ab718 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:59.437231 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-817d7a4e-ccad-4f40-938a-5500040ab718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:59.456643 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:15:59.555977 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fbdcaa-a251-46da-b4bc-13fbd31d986f None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:59.556277 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fbdcaa-a251-46da-b4bc-13fbd31d986f None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:59.556616 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fbdcaa-a251-46da-b4bc-13fbd31d986f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:59.559184 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1221/2438] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:15:59.586408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-d2fbdcaa-a251-46da-b4bc-13fbd31d986f None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:15:59.592600 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d2fbdcaa-a251-46da-b4bc-13fbd31d986f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:15:59.605702 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1218/2439] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:15:59.615246 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d7a4e-ccad-4f40-938a-5500040ab718 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:59.621840 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d7a4e-ccad-4f40-938a-5500040ab718 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:59.637362 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69843703-d9e4-4052-aa00-9375b131ef16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:59.638486 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69843703-d9e4-4052-aa00-9375b131ef16 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:59.638941 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69843703-d9e4-4052-aa00-9375b131ef16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:59.639293 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69843703-d9e4-4052-aa00-9375b131ef16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:59.640375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69843703-d9e4-4052-aa00-9375b131ef16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:15:59.640768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69843703-d9e4-4052-aa00-9375b131ef16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:59.644006 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69843703-d9e4-4052-aa00-9375b131ef16 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:59.684317 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1222/2440] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:15:59.686054 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fbdcaa-a251-46da-b4bc-13fbd31d986f None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:59.697328 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fbdcaa-a251-46da-b4bc-13fbd31d986f None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:15:59.704737 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8f3b6-0db1-4a35-b883-72dcf280bdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:15:59.705501 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8f3b6-0db1-4a35-b883-72dcf280bdc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:15:59.705786 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8f3b6-0db1-4a35-b883-72dcf280bdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:15:59.706229 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8f3b6-0db1-4a35-b883-72dcf280bdc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:15:59.706741 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e8f3b6-0db1-4a35-b883-72dcf280bdc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:15:59.708899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e8f3b6-0db1-4a35-b883-72dcf280bdc3 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:15:59.829550 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69843703-d9e4-4052-aa00-9375b131ef16 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:15:59.839031 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1219/2441] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:15:59 2022] POST /identity/v3/projects => generated 371 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:15:59.844876 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69843703-d9e4-4052-aa00-9375b131ef16 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:16:00.004007 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a596461-09ac-431f-8b06-4dddcf0a83f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:16:00.005701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a596461-09ac-431f-8b06-4dddcf0a83f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:00.006506 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a596461-09ac-431f-8b06-4dddcf0a83f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:00.007040 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a596461-09ac-431f-8b06-4dddcf0a83f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:00.008291 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a596461-09ac-431f-8b06-4dddcf0a83f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:16:00.011074 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a596461-09ac-431f-8b06-4dddcf0a83f7 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:16:00.106463 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8f3b6-0db1-4a35-b883-72dcf280bdc3 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:16:00.113342 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1223/2442] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:15:59 2022] POST /identity/v3/users => generated 286 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:16:00.122099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8f3b6-0db1-4a35-b883-72dcf280bdc3 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:16:00.129323 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb955cf-54c8-4c99-a4c1-ffbb3608643a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:16:00.129968 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb955cf-54c8-4c99-a4c1-ffbb3608643a admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:00.134020 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb955cf-54c8-4c99-a4c1-ffbb3608643a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:00.134020 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb955cf-54c8-4c99-a4c1-ffbb3608643a admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:00.134020 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb955cf-54c8-4c99-a4c1-ffbb3608643a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:16:00.134020 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb955cf-54c8-4c99-a4c1-ffbb3608643a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:16:00.161870 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1220/2443] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:16:00.168405 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a596461-09ac-431f-8b06-4dddcf0a83f7 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:16:00.188656 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a596461-09ac-431f-8b06-4dddcf0a83f7 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:16:00.196310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb3ebb2-cab6-4dc7-bede-a07b819d8d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:16:00.197421 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb3ebb2-cab6-4dc7-bede-a07b819d8d0b admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:00.197682 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb3ebb2-cab6-4dc7-bede-a07b819d8d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:00.198577 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb3ebb2-cab6-4dc7-bede-a07b819d8d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:00.199368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb3ebb2-cab6-4dc7-bede-a07b819d8d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:16:00.201148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb3ebb2-cab6-4dc7-bede-a07b819d8d0b admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:16:00.215088 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1224/2444] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:16:00.219634 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb955cf-54c8-4c99-a4c1-ffbb3608643a admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:16:00.234706 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb955cf-54c8-4c99-a4c1-ffbb3608643a admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:16:00.241268 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ef2f2-4c4c-47bb-9859-58c2dbd0a989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:16:00.244693 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ef2f2-4c4c-47bb-9859-58c2dbd0a989 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:00.245008 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ef2f2-4c4c-47bb-9859-58c2dbd0a989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:00.245297 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ef2f2-4c4c-47bb-9859-58c2dbd0a989 admin admin] PATH_INFO: `/v3/projects/3ecf0d3c745b4b0cb16ad6d0f60d0460/users/2975cd59b8934090a03053cd8618a0e8/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:00.279253 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ef2f2-4c4c-47bb-9859-58c2dbd0a989 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ecf0d3c745b4b0cb16ad6d0f60d0460, user_id=2975cd59b8934090a03053cd8618a0e8, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:16:00.283141 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ef2f2-4c4c-47bb-9859-58c2dbd0a989 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:16:00.410383 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1221/2445] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:16:00 2022] PUT /identity/v3/projects/3ecf0d3c745b4b0cb16ad6d0f60d0460/users/2975cd59b8934090a03053cd8618a0e8/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:16:00.424940 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d7af9-95b7-4a30-b5f6-09124074d3bd None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:00.425318 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d7af9-95b7-4a30-b5f6-09124074d3bd None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:00.425520 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d7af9-95b7-4a30-b5f6-09124074d3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:00.462099 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-d47d7af9-95b7-4a30-b5f6-09124074d3bd None None] MFA Rules not processed for user `2975cd59b8934090a03053cd8618a0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:16:00.465120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-d47d7af9-95b7-4a30-b5f6-09124074d3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:16:00.598115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1225/2446] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:16:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:16:00.612110 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ef2f2-4c4c-47bb-9859-58c2dbd0a989 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:16:00.614063 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a37ef2f2-4c4c-47bb-9859-58c2dbd0a989 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:16:00.675496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:16:00.743131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ef2f2-4c4c-47bb-9859-58c2dbd0a989 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:16:00.751950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0594df1e-02a2-457a-a77e-9527dca712f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:16:00.752950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594df1e-02a2-457a-a77e-9527dca712f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:00.753433 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594df1e-02a2-457a-a77e-9527dca712f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:00.753886 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594df1e-02a2-457a-a77e-9527dca712f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:00.777897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0594df1e-02a2-457a-a77e-9527dca712f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:16:00.779516 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0594df1e-02a2-457a-a77e-9527dca712f2 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:16:00.812304 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1222/2447] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:16:00 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:16:00.840485 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d7af9-95b7-4a30-b5f6-09124074d3bd None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:16:00.874905 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:16:00.967426 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d7af9-95b7-4a30-b5f6-09124074d3bd None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:16:00.972842 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82170678-4380-4b11-80f7-c54cdb94825d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:16:00.973656 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82170678-4380-4b11-80f7-c54cdb94825d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:00.974118 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82170678-4380-4b11-80f7-c54cdb94825d admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:00.974484 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82170678-4380-4b11-80f7-c54cdb94825d admin admin] PATH_INFO: `/v3/users/2975cd59b8934090a03053cd8618a0e8` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:00.975846 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82170678-4380-4b11-80f7-c54cdb94825d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2975cd59b8934090a03053cd8618a0e8)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:16:00.983169 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82170678-4380-4b11-80f7-c54cdb94825d admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:16:01.205989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-82170678-4380-4b11-80f7-c54cdb94825d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2975cd59b8934090a03053cd8618a0e8'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:16:01.293816 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-82170678-4380-4b11-80f7-c54cdb94825d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2975cd59b8934090a03053cd8618a0e8'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:16:01.294887 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-82170678-4380-4b11-80f7-c54cdb94825d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2975cd59b8934090a03053cd8618a0e8'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:16:01.309568 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-82170678-4380-4b11-80f7-c54cdb94825d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2975cd59b8934090a03053cd8618a0e8'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:16:01.310505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-82170678-4380-4b11-80f7-c54cdb94825d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2975cd59b8934090a03053cd8618a0e8'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:16:01.324639 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1226/2448] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:16:00 2022] DELETE /identity/v3/users/2975cd59b8934090a03053cd8618a0e8 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:16:01.329424 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a731b19-47ef-4760-b148-c4461794071f None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:01.330177 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a731b19-47ef-4760-b148-c4461794071f None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:01.330177 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a731b19-47ef-4760-b148-c4461794071f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:01.351657 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-0a731b19-47ef-4760-b148-c4461794071f None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:16:01.352902 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0a731b19-47ef-4760-b148-c4461794071f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:16:01.423904 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1223/2449] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:16:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:16:01.444259 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82170678-4380-4b11-80f7-c54cdb94825d admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:16:01.446100 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-82170678-4380-4b11-80f7-c54cdb94825d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:16:01.527231 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82170678-4380-4b11-80f7-c54cdb94825d admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:16:01.536838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ff133-9432-4957-85bd-e9bf0dc87729 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:16:01.541326 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ff133-9432-4957-85bd-e9bf0dc87729 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:01.541326 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ff133-9432-4957-85bd-e9bf0dc87729 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:01.541326 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ff133-9432-4957-85bd-e9bf0dc87729 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:01.555755 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ff133-9432-4957-85bd-e9bf0dc87729 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:16:01.555755 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ff133-9432-4957-85bd-e9bf0dc87729 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:16:01.576802 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1227/2450] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:16:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:16:02.784617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a731b19-47ef-4760-b148-c4461794071f None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:16:02.787534 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0a731b19-47ef-4760-b148-c4461794071f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:16:02.802536 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a731b19-47ef-4760-b148-c4461794071f None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:16:02.809667 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f60ed-15cd-4f16-aedb-79247e44fadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:16:02.810655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f60ed-15cd-4f16-aedb-79247e44fadf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:02.811255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f60ed-15cd-4f16-aedb-79247e44fadf admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:02.811728 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f60ed-15cd-4f16-aedb-79247e44fadf admin admin] PATH_INFO: `/v3/projects/3ecf0d3c745b4b0cb16ad6d0f60d0460` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:02.813378 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f60ed-15cd-4f16-aedb-79247e44fadf admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ecf0d3c745b4b0cb16ad6d0f60d0460)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:16:02.816267 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f60ed-15cd-4f16-aedb-79247e44fadf admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:16:02.981682 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-be4f60ed-15cd-4f16-aedb-79247e44fadf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ecf0d3c745b4b0cb16ad6d0f60d0460', 'request_id': 'req-be4f60ed-15cd-4f16-aedb-79247e44fadf'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:16:03.101886 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-be4f60ed-15cd-4f16-aedb-79247e44fadf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ecf0d3c745b4b0cb16ad6d0f60d0460', 'request_id': 'req-be4f60ed-15cd-4f16-aedb-79247e44fadf'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:16:03.111437 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-be4f60ed-15cd-4f16-aedb-79247e44fadf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ecf0d3c745b4b0cb16ad6d0f60d0460', 'request_id': 'req-be4f60ed-15cd-4f16-aedb-79247e44fadf'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:16:03.125953 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-be4f60ed-15cd-4f16-aedb-79247e44fadf admin admin] The token cache is being invalidate because project 3ecf0d3c745b4b0cb16ad6d0f60d0460 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:16:03.126251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-be4f60ed-15cd-4f16-aedb-79247e44fadf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:16:03.127183 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-be4f60ed-15cd-4f16-aedb-79247e44fadf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:16:03.135954 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1224/2451] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:16:02 2022] DELETE /identity/v3/projects/3ecf0d3c745b4b0cb16ad6d0f60d0460 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:16:03.140906 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd3c23-567f-4e04-bc93-1d214f03a809 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:03.141471 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd3c23-567f-4e04-bc93-1d214f03a809 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:03.141471 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd3c23-567f-4e04-bc93-1d214f03a809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:03.163819 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-99dd3c23-567f-4e04-bc93-1d214f03a809 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:16:03.165361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-99dd3c23-567f-4e04-bc93-1d214f03a809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:16:03.225224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:16:03.258449 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1228/2452] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:16:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:16:03.333247 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56715954-5bee-4f90-b379-c404aba71524 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:03.333599 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56715954-5bee-4f90-b379-c404aba71524 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:03.333980 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56715954-5bee-4f90-b379-c404aba71524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:03.375028 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-56715954-5bee-4f90-b379-c404aba71524 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:16:03.376247 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-56715954-5bee-4f90-b379-c404aba71524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:16:03.390483 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1225/2453] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:16:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:16:03.395163 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd3c23-567f-4e04-bc93-1d214f03a809 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:16:03.413839 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd3c23-567f-4e04-bc93-1d214f03a809 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:16:03.421286 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc01836-b52b-4651-9fdb-6c0a9efe2047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:16:03.421921 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc01836-b52b-4651-9fdb-6c0a9efe2047 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:03.422206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc01836-b52b-4651-9fdb-6c0a9efe2047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:03.422739 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc01836-b52b-4651-9fdb-6c0a9efe2047 admin admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:03.423557 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc01836-b52b-4651-9fdb-6c0a9efe2047 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:16:03.423922 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc01836-b52b-4651-9fdb-6c0a9efe2047 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:16:03.425627 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc01836-b52b-4651-9fdb-6c0a9efe2047 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:16:03.440342 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1229/2454] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:16:03.444997 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56715954-5bee-4f90-b379-c404aba71524 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:16:03.460563 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56715954-5bee-4f90-b379-c404aba71524 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:16:03.474597 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c1bbe-2589-480b-8a40-ce9bce693c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:16:03.474597 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c1bbe-2589-480b-8a40-ce9bce693c77 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:03.474597 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c1bbe-2589-480b-8a40-ce9bce693c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:03.474597 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c1bbe-2589-480b-8a40-ce9bce693c77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:03.475226 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c1bbe-2589-480b-8a40-ce9bce693c77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:16:03.475226 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c1bbe-2589-480b-8a40-ce9bce693c77 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:16:03.628649 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1226/2455] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:16:03 2022] POST /identity/v3/projects => generated 381 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:16:03.632579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc01836-b52b-4651-9fdb-6c0a9efe2047 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:16:03.645143 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc01836-b52b-4651-9fdb-6c0a9efe2047 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:16:03.736230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76249e6-ba0c-4211-bada-a1ce03fd7cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:16:03.737339 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76249e6-ba0c-4211-bada-a1ce03fd7cef admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:03.738550 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76249e6-ba0c-4211-bada-a1ce03fd7cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:03.739151 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76249e6-ba0c-4211-bada-a1ce03fd7cef admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:03.740076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76249e6-ba0c-4211-bada-a1ce03fd7cef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:16:03.742646 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76249e6-ba0c-4211-bada-a1ce03fd7cef admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:16:03.873035 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1230/2456] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:16:03 2022] POST /identity/v3/users => generated 291 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:16:03.874537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c1bbe-2589-480b-8a40-ce9bce693c77 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:16:03.886321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c1bbe-2589-480b-8a40-ce9bce693c77 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:16:03.892153 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a046a2f-bb92-4df1-ba3b-53d6895fcc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:16:03.893792 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a046a2f-bb92-4df1-ba3b-53d6895fcc00 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:03.894171 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a046a2f-bb92-4df1-ba3b-53d6895fcc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:03.894531 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a046a2f-bb92-4df1-ba3b-53d6895fcc00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:03.895367 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a046a2f-bb92-4df1-ba3b-53d6895fcc00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:16:03.897232 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a046a2f-bb92-4df1-ba3b-53d6895fcc00 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:16:03.915325 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1227/2457] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:16:03.919909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76249e6-ba0c-4211-bada-a1ce03fd7cef admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:16:03.931460 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76249e6-ba0c-4211-bada-a1ce03fd7cef admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:16:03.937211 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce7fcf-8902-4dc4-aa43-cf3e43c56725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:16:03.941850 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce7fcf-8902-4dc4-aa43-cf3e43c56725 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:03.941850 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce7fcf-8902-4dc4-aa43-cf3e43c56725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:03.941850 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce7fcf-8902-4dc4-aa43-cf3e43c56725 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:03.941850 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce7fcf-8902-4dc4-aa43-cf3e43c56725 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:16:03.941850 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce7fcf-8902-4dc4-aa43-cf3e43c56725 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:16:03.952224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1231/2458] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:16:03.957166 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a046a2f-bb92-4df1-ba3b-53d6895fcc00 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:16:03.968138 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a046a2f-bb92-4df1-ba3b-53d6895fcc00 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:16:03.976725 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f9ed1-152a-42a1-847f-db53967cd99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:16:03.976725 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f9ed1-152a-42a1-847f-db53967cd99b admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:03.976725 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f9ed1-152a-42a1-847f-db53967cd99b admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:03.976725 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f9ed1-152a-42a1-847f-db53967cd99b admin admin] PATH_INFO: `/v3/projects/6675985e19c44dabb624420aaa97b0f7/users/28d7d91983dc4c0fb218e7bdbc1c6558/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:03.990956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f9ed1-152a-42a1-847f-db53967cd99b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6675985e19c44dabb624420aaa97b0f7, user_id=28d7d91983dc4c0fb218e7bdbc1c6558, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:16:03.993624 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f9ed1-152a-42a1-847f-db53967cd99b admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:16:03.998164 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:16:04.070897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1228/2459] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:16:03 2022] PUT /identity/v3/projects/6675985e19c44dabb624420aaa97b0f7/users/28d7d91983dc4c0fb218e7bdbc1c6558/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:16:04.076537 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d88d06-4d38-4f60-b8bf-d14cbcacc330 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:04.076537 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d88d06-4d38-4f60-b8bf-d14cbcacc330 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:04.076839 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d88d06-4d38-4f60-b8bf-d14cbcacc330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:04.096951 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-51d88d06-4d38-4f60-b8bf-d14cbcacc330 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:16:04.098192 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-51d88d06-4d38-4f60-b8bf-d14cbcacc330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:16:04.172433 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1232/2460] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:16:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:16:04.181755 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f9ed1-152a-42a1-847f-db53967cd99b admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:16:04.184188 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9e7f9ed1-152a-42a1-847f-db53967cd99b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:16:04.251608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f9ed1-152a-42a1-847f-db53967cd99b admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:16:04.256608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e8c1f-56d8-463d-89f9-db0abe45231c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:16:04.257326 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e8c1f-56d8-463d-89f9-db0abe45231c service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:04.257681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e8c1f-56d8-463d-89f9-db0abe45231c service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:04.258033 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e8c1f-56d8-463d-89f9-db0abe45231c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:04.269072 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e8c1f-56d8-463d-89f9-db0abe45231c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:16:04.270787 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e8c1f-56d8-463d-89f9-db0abe45231c service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:16:04.282269 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1229/2461] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:16:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:16:05.685543 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82448c17-3189-4c70-b68f-dc6569e5b15d None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:05.691398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82448c17-3189-4c70-b68f-dc6569e5b15d None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:05.691398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82448c17-3189-4c70-b68f-dc6569e5b15d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:05.735295 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-82448c17-3189-4c70-b68f-dc6569e5b15d None None] MFA Rules not processed for user `28d7d91983dc4c0fb218e7bdbc1c6558`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:16:05.737094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-82448c17-3189-4c70-b68f-dc6569e5b15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:16:05.805205 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:16:05.881768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1233/2462] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:16:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:16:05.893931 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e8c1f-56d8-463d-89f9-db0abe45231c service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:16:05.900677 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-361e8c1f-56d8-463d-89f9-db0abe45231c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:16:06.023861 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e8c1f-56d8-463d-89f9-db0abe45231c service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:16:06.038517 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c81e8e-aafc-4100-a058-9ab75410e9ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:16:06.039189 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c81e8e-aafc-4100-a058-9ab75410e9ad service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:06.039479 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c81e8e-aafc-4100-a058-9ab75410e9ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:06.039739 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c81e8e-aafc-4100-a058-9ab75410e9ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:06.053007 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c81e8e-aafc-4100-a058-9ab75410e9ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:16:06.053904 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c81e8e-aafc-4100-a058-9ab75410e9ad service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:16:06.079883 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1230/2463] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:16:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:16:07.996155 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f47f3a-18b6-432d-9d02-44057286aed1 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:07.999026 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f47f3a-18b6-432d-9d02-44057286aed1 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:07.999026 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f47f3a-18b6-432d-9d02-44057286aed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:08.016563 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-49f47f3a-18b6-432d-9d02-44057286aed1 None None] MFA Rules not processed for user `28d7d91983dc4c0fb218e7bdbc1c6558`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:16:08.017980 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-49f47f3a-18b6-432d-9d02-44057286aed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:16:08.026587 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1234/2464] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:16:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:16:16.062773 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=54 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:16:18.011425 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=14 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:16:31.663088 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02c551-5f1c-4014-ba1e-ebe99ef5565c None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:31.663584 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02c551-5f1c-4014-ba1e-ebe99ef5565c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:31.664396 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02c551-5f1c-4014-ba1e-ebe99ef5565c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:31.685885 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-6a02c551-5f1c-4014-ba1e-ebe99ef5565c None None] MFA Rules not processed for user `de853a2dfffb4f05b6f47df2280ae4e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:16:31.687972 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6a02c551-5f1c-4014-ba1e-ebe99ef5565c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:16:31.751698 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1231/2465] 10.222.0.40 () {58 vars in 967 bytes} [Wed Aug 3 05:16:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:16:31.781697 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f47f3a-18b6-432d-9d02-44057286aed1 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:16:31.784048 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-49f47f3a-18b6-432d-9d02-44057286aed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:16:31.867962 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f47f3a-18b6-432d-9d02-44057286aed1 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:16:31.873606 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935758a5-c3f7-4fec-a9b2-edd1fc701718 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1014bcb4d36441aa46042941c9a63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:16:31.876761 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935758a5-c3f7-4fec-a9b2-edd1fc701718 service swift] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:31.876761 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935758a5-c3f7-4fec-a9b2-edd1fc701718 service swift] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:31.876761 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935758a5-c3f7-4fec-a9b2-edd1fc701718 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:31.883485 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935758a5-c3f7-4fec-a9b2-edd1fc701718 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:16:31.885625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935758a5-c3f7-4fec-a9b2-edd1fc701718 service swift] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:16:31.895679 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1235/2466] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 3 05:16:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 05:16:32.273439 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02c551-5f1c-4014-ba1e-ebe99ef5565c None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:16:32.288285 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02c551-5f1c-4014-ba1e-ebe99ef5565c None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:16:32.295340 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22bee6a-f77d-4c35-804d-8f364aac7013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:16:32.296077 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22bee6a-f77d-4c35-804d-8f364aac7013 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:32.296486 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22bee6a-f77d-4c35-804d-8f364aac7013 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:32.296865 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22bee6a-f77d-4c35-804d-8f364aac7013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:32.299226 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c22bee6a-f77d-4c35-804d-8f364aac7013 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:16:32.378333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22bee6a-f77d-4c35-804d-8f364aac7013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:16:32.379838 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22bee6a-f77d-4c35-804d-8f364aac7013 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:16:32.390347 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1232/2467] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:16:32 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:16:41.833860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e510d-1825-4a58-a8f0-0ab91cb7d2b4 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:41.834334 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e510d-1825-4a58-a8f0-0ab91cb7d2b4 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:41.834431 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e510d-1825-4a58-a8f0-0ab91cb7d2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:41.857410 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-af2e510d-1825-4a58-a8f0-0ab91cb7d2b4 None None] MFA Rules not processed for user `de853a2dfffb4f05b6f47df2280ae4e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:16:41.859024 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-af2e510d-1825-4a58-a8f0-0ab91cb7d2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:16:41.867741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1236/2468] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Aug 3 05:16:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:16:41.879803 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22bee6a-f77d-4c35-804d-8f364aac7013 service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:16:41.892858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22bee6a-f77d-4c35-804d-8f364aac7013 service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:16:41.900393 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5303f-2d8a-4be8-8860-3bc58e0a13fb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1014bcb4d36441aa46042941c9a63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:16:41.901021 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5303f-2d8a-4be8-8860-3bc58e0a13fb service swift] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:41.901304 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5303f-2d8a-4be8-8860-3bc58e0a13fb service swift] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:41.901582 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5303f-2d8a-4be8-8860-3bc58e0a13fb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:41.910800 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5303f-2d8a-4be8-8860-3bc58e0a13fb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:16:41.912511 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5303f-2d8a-4be8-8860-3bc58e0a13fb service swift] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:16:41.921658 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1233/2469] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 3 05:16:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 05:16:42.167707 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e510d-1825-4a58-a8f0-0ab91cb7d2b4 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:16:42.169138 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-af2e510d-1825-4a58-a8f0-0ab91cb7d2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:16:42.180862 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e510d-1825-4a58-a8f0-0ab91cb7d2b4 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:16:42.185713 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6eb11b-883c-4738-8948-bd59298140b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:16:42.186425 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6eb11b-883c-4738-8948-bd59298140b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:42.186762 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6eb11b-883c-4738-8948-bd59298140b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:42.187073 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6eb11b-883c-4738-8948-bd59298140b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:42.189230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-1a6eb11b-883c-4738-8948-bd59298140b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:16:42.254904 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6eb11b-883c-4738-8948-bd59298140b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:16:42.256390 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6eb11b-883c-4738-8948-bd59298140b7 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:16:42.266519 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1237/2470] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:16:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:16:42.888499 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b65e8-270b-42b4-9205-bb25c0b6e8a2 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:42.889245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b65e8-270b-42b4-9205-bb25c0b6e8a2 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:42.889245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b65e8-270b-42b4-9205-bb25c0b6e8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:42.914825 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-220b65e8-270b-42b4-9205-bb25c0b6e8a2 None None] MFA Rules not processed for user `de853a2dfffb4f05b6f47df2280ae4e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:16:42.915958 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-220b65e8-270b-42b4-9205-bb25c0b6e8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:16:42.923812 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1234/2471] 10.222.0.40 () {58 vars in 967 bytes} [Wed Aug 3 05:16:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:16:42.937130 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6eb11b-883c-4738-8948-bd59298140b7 service neutron] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:16:42.948741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6eb11b-883c-4738-8948-bd59298140b7 service neutron] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:16:42.953995 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68cb9f9-2058-4854-ba55-c67785a97244 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1014bcb4d36441aa46042941c9a63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:16:42.954612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68cb9f9-2058-4854-ba55-c67785a97244 service swift] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:42.954926 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68cb9f9-2058-4854-ba55-c67785a97244 service swift] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:42.955208 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68cb9f9-2058-4854-ba55-c67785a97244 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:42.964122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68cb9f9-2058-4854-ba55-c67785a97244 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:16:42.965805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68cb9f9-2058-4854-ba55-c67785a97244 service swift] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:16:42.975544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1238/2472] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 3 05:16:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 05:16:43.678639 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa95de1-81f6-4168-9707-006e227e036d None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:43.678965 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa95de1-81f6-4168-9707-006e227e036d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:43.679250 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa95de1-81f6-4168-9707-006e227e036d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:43.698752 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-efa95de1-81f6-4168-9707-006e227e036d None None] MFA Rules not processed for user `de853a2dfffb4f05b6f47df2280ae4e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:16:43.700431 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-efa95de1-81f6-4168-9707-006e227e036d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:16:43.713306 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1235/2473] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Aug 3 05:16:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:16:43.721997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68cb9f9-2058-4854-ba55-c67785a97244 service swift] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:16:43.734680 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68cb9f9-2058-4854-ba55-c67785a97244 service swift] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:16:43.741416 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325131a-3fbd-4660-b1a8-b8f63719cfcf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1014bcb4d36441aa46042941c9a63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:16:43.742323 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325131a-3fbd-4660-b1a8-b8f63719cfcf service swift] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:43.742612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325131a-3fbd-4660-b1a8-b8f63719cfcf service swift] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:43.742919 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325131a-3fbd-4660-b1a8-b8f63719cfcf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:43.756838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3325131a-3fbd-4660-b1a8-b8f63719cfcf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:16:43.758569 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3325131a-3fbd-4660-b1a8-b8f63719cfcf service swift] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:16:43.767988 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1239/2474] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 3 05:16:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 05:16:44.504039 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868310ee-758d-4386-82cd-5f383beb81b2 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:44.504576 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868310ee-758d-4386-82cd-5f383beb81b2 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:44.504576 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868310ee-758d-4386-82cd-5f383beb81b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:44.526481 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-868310ee-758d-4386-82cd-5f383beb81b2 None None] MFA Rules not processed for user `de853a2dfffb4f05b6f47df2280ae4e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:16:44.528086 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-868310ee-758d-4386-82cd-5f383beb81b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:16:44.543195 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1236/2475] 10.222.0.40 () {58 vars in 967 bytes} [Wed Aug 3 05:16:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:16:44.577502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325131a-3fbd-4660-b1a8-b8f63719cfcf service swift] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:16:44.592358 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325131a-3fbd-4660-b1a8-b8f63719cfcf service swift] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:16:44.599424 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908aa1f-22f7-440e-82b0-a74015ad6cff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1014bcb4d36441aa46042941c9a63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:16:44.600055 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908aa1f-22f7-440e-82b0-a74015ad6cff service swift] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:44.600334 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908aa1f-22f7-440e-82b0-a74015ad6cff service swift] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:44.600608 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908aa1f-22f7-440e-82b0-a74015ad6cff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:44.617126 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a908aa1f-22f7-440e-82b0-a74015ad6cff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:16:44.620703 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a908aa1f-22f7-440e-82b0-a74015ad6cff service swift] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:16:44.651693 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1240/2476] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 3 05:16:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 05:16:47.733755 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868310ee-758d-4386-82cd-5f383beb81b2 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:16:47.735382 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-868310ee-758d-4386-82cd-5f383beb81b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:16:47.751283 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868310ee-758d-4386-82cd-5f383beb81b2 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:16:47.756471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29666f0-0f96-4312-9af8-b5b60667f347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:16:47.757562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29666f0-0f96-4312-9af8-b5b60667f347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:47.757562 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29666f0-0f96-4312-9af8-b5b60667f347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:47.758482 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29666f0-0f96-4312-9af8-b5b60667f347 admin admin] PATH_INFO: `/v3/users/ff504c0570b24f3baa8744a9740bcfce` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:47.760048 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29666f0-0f96-4312-9af8-b5b60667f347 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff504c0570b24f3baa8744a9740bcfce)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:16:47.762475 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29666f0-0f96-4312-9af8-b5b60667f347 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:16:47.957018 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b29666f0-0f96-4312-9af8-b5b60667f347 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff504c0570b24f3baa8744a9740bcfce'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:16:48.039937 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b29666f0-0f96-4312-9af8-b5b60667f347 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff504c0570b24f3baa8744a9740bcfce'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:16:48.040683 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b29666f0-0f96-4312-9af8-b5b60667f347 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff504c0570b24f3baa8744a9740bcfce'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:16:48.053819 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:16:48.073433 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b29666f0-0f96-4312-9af8-b5b60667f347 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff504c0570b24f3baa8744a9740bcfce'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:16:48.074307 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b29666f0-0f96-4312-9af8-b5b60667f347 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff504c0570b24f3baa8744a9740bcfce'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:16:48.094759 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1237/2477] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:16:47 2022] DELETE /identity/v3/users/ff504c0570b24f3baa8744a9740bcfce => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:16:48.099299 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd3e72-f481-495e-a390-ae7d5938069a None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:48.100913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd3e72-f481-495e-a390-ae7d5938069a None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:48.101381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd3e72-f481-495e-a390-ae7d5938069a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:48.124018 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-eedd3e72-f481-495e-a390-ae7d5938069a None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:16:48.125278 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-eedd3e72-f481-495e-a390-ae7d5938069a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:16:48.163328 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:16:48.196011 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1241/2478] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:16:48.206915 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29666f0-0f96-4312-9af8-b5b60667f347 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:16:48.208908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b29666f0-0f96-4312-9af8-b5b60667f347 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:16:48.327606 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29666f0-0f96-4312-9af8-b5b60667f347 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:16:48.339851 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02678c43-2a82-4bdc-b5eb-d2414075b43b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:16:48.340990 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02678c43-2a82-4bdc-b5eb-d2414075b43b service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:48.340990 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02678c43-2a82-4bdc-b5eb-d2414075b43b service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:48.341293 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02678c43-2a82-4bdc-b5eb-d2414075b43b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:48.351329 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02678c43-2a82-4bdc-b5eb-d2414075b43b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:16:48.352892 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02678c43-2a82-4bdc-b5eb-d2414075b43b service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:16:48.372768 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1238/2479] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:16:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:16:48.702112 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd3e72-f481-495e-a390-ae7d5938069a None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:16:48.720042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-eedd3e72-f481-495e-a390-ae7d5938069a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:16:48.743074 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd3e72-f481-495e-a390-ae7d5938069a None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:16:48.750130 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e87ef7-e9e9-4b78-b327-6cae5ff7b75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:16:48.751123 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e87ef7-e9e9-4b78-b327-6cae5ff7b75d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:16:48.751604 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e87ef7-e9e9-4b78-b327-6cae5ff7b75d admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:16:48.752116 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e87ef7-e9e9-4b78-b327-6cae5ff7b75d admin admin] PATH_INFO: `/v3/projects/7a970d4cbb3545a187c75848e10cb166` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:16:48.764116 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e87ef7-e9e9-4b78-b327-6cae5ff7b75d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a970d4cbb3545a187c75848e10cb166)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:16:48.766655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e87ef7-e9e9-4b78-b327-6cae5ff7b75d admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:16:48.913292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-87e87ef7-e9e9-4b78-b327-6cae5ff7b75d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a970d4cbb3545a187c75848e10cb166', 'request_id': 'req-87e87ef7-e9e9-4b78-b327-6cae5ff7b75d'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:16:48.922924 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-87e87ef7-e9e9-4b78-b327-6cae5ff7b75d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a970d4cbb3545a187c75848e10cb166', 'request_id': 'req-87e87ef7-e9e9-4b78-b327-6cae5ff7b75d'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:16:48.999368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-87e87ef7-e9e9-4b78-b327-6cae5ff7b75d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a970d4cbb3545a187c75848e10cb166', 'request_id': 'req-87e87ef7-e9e9-4b78-b327-6cae5ff7b75d'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:16:49.011604 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-87e87ef7-e9e9-4b78-b327-6cae5ff7b75d admin admin] The token cache is being invalidate because project 7a970d4cbb3545a187c75848e10cb166 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:16:49.011898 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-87e87ef7-e9e9-4b78-b327-6cae5ff7b75d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:16:49.012541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-87e87ef7-e9e9-4b78-b327-6cae5ff7b75d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:16:49.014154 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1242/2480] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:16:48 2022] DELETE /identity/v3/projects/7a970d4cbb3545a187c75848e10cb166 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:16:58.473130 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=35,keystone:DELETE=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:16:59.007711 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=37,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:17:21.107040 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c6691-c539-452f-8dd2-97ac11f9d7f0 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:17:21.107575 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c6691-c539-452f-8dd2-97ac11f9d7f0 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:17:21.107678 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c6691-c539-452f-8dd2-97ac11f9d7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:17:21.126472 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-ee5c6691-c539-452f-8dd2-97ac11f9d7f0 None None] MFA Rules not processed for user `bb0fe4cb98e0447cad3e126da1a8de9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:17:21.127741 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ee5c6691-c539-452f-8dd2-97ac11f9d7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:17:21.187435 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1239/2481] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:17:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:17:31.180555 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=25 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:17:41.679572 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e55a0f-7902-42b4-85bf-3a2f4f4ad495 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:17:41.684409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e55a0f-7902-42b4-85bf-3a2f4f4ad495 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:17:41.684409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e55a0f-7902-42b4-85bf-3a2f4f4ad495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:17:41.699676 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-f9e55a0f-7902-42b4-85bf-3a2f4f4ad495 None None] MFA Rules not processed for user `28d7d91983dc4c0fb218e7bdbc1c6558`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:17:41.700890 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-f9e55a0f-7902-42b4-85bf-3a2f4f4ad495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:17:41.756344 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1243/2482] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:17:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:17:49.951324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c6691-c539-452f-8dd2-97ac11f9d7f0 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:17:49.953574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ee5c6691-c539-452f-8dd2-97ac11f9d7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:17:50.063080 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c6691-c539-452f-8dd2-97ac11f9d7f0 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:17:50.068112 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07840fe-6dac-4d67-b47d-67292db6f447 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:17:50.069626 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07840fe-6dac-4d67-b47d-67292db6f447 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:17:50.070100 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07840fe-6dac-4d67-b47d-67292db6f447 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:17:50.070576 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07840fe-6dac-4d67-b47d-67292db6f447 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:17:50.073235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f07840fe-6dac-4d67-b47d-67292db6f447 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:17:50.173032 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07840fe-6dac-4d67-b47d-67292db6f447 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:17:50.174705 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07840fe-6dac-4d67-b47d-67292db6f447 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:17:50.192049 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1240/2483] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:17:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:17:51.749359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=25 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:17:51.939908 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ef5cf-5450-4716-a320-bb55e9888e5b None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:17:51.940121 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ef5cf-5450-4716-a320-bb55e9888e5b None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:17:51.940403 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ef5cf-5450-4716-a320-bb55e9888e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:17:51.963296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-bb3ef5cf-5450-4716-a320-bb55e9888e5b None None] MFA Rules not processed for user `747a608061504d7693f0f99ecdfa476e`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:17:51.968304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-bb3ef5cf-5450-4716-a320-bb55e9888e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:17:52.046894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1244/2484] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:17:51 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:17:52.643565 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae559af-5520-4184-a83e-e22cb55db678 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:17:52.644327 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae559af-5520-4184-a83e-e22cb55db678 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:17:52.644709 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae559af-5520-4184-a83e-e22cb55db678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:17:52.665470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-3ae559af-5520-4184-a83e-e22cb55db678 None None] MFA Rules not processed for user `f01737ba3200470c8635f335d0338ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:17:52.666771 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3ae559af-5520-4184-a83e-e22cb55db678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:17:52.732189 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1241/2485] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:17:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:17:53.326665 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ef5cf-5450-4716-a320-bb55e9888e5b None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:17:53.330441 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-bb3ef5cf-5450-4716-a320-bb55e9888e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:17:53.404359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ef5cf-5450-4716-a320-bb55e9888e5b None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:17:53.408112 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1b8bf-3bf2-491e-9e01-208587036161 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cc14b33895443b893fab487bfe1858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:17:53.408744 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1b8bf-3bf2-491e-9e01-208587036161 service glance] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:17:53.409280 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1b8bf-3bf2-491e-9e01-208587036161 service glance] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:17:53.409557 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1b8bf-3bf2-491e-9e01-208587036161 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:17:53.411559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-aca1b8bf-3bf2-491e-9e01-208587036161 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:17:53.427054 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca1b8bf-3bf2-491e-9e01-208587036161 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:17:53.428590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca1b8bf-3bf2-491e-9e01-208587036161 service glance] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:17:53.441442 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1245/2486] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:17:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:17:59.704051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae559af-5520-4184-a83e-e22cb55db678 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:17:59.705500 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3ae559af-5520-4184-a83e-e22cb55db678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:17:59.722480 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae559af-5520-4184-a83e-e22cb55db678 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:17:59.729431 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add883d7-7fa5-4efa-b791-19f226f8f452 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:17:59.731691 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add883d7-7fa5-4efa-b791-19f226f8f452 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:17:59.732341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add883d7-7fa5-4efa-b791-19f226f8f452 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:17:59.732341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add883d7-7fa5-4efa-b791-19f226f8f452 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:17:59.734530 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-add883d7-7fa5-4efa-b791-19f226f8f452 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:17:59.749161 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add883d7-7fa5-4efa-b791-19f226f8f452 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:17:59.751627 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add883d7-7fa5-4efa-b791-19f226f8f452 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:17:59.771417 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1242/2487] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:17:59 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:18:03.435866 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=51 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:18:05.185068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1b8bf-3bf2-491e-9e01-208587036161 service glance] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:18:05.196348 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1b8bf-3bf2-491e-9e01-208587036161 service glance] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:18:05.202549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7e667-5d1d-4fa6-bd51-592e611f682f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:18:05.203200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7e667-5d1d-4fa6-bd51-592e611f682f service nova] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:18:05.203489 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7e667-5d1d-4fa6-bd51-592e611f682f service nova] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:18:05.203928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7e667-5d1d-4fa6-bd51-592e611f682f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:18:05.205673 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-abe7e667-5d1d-4fa6-bd51-592e611f682f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:18:05.217756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe7e667-5d1d-4fa6-bd51-592e611f682f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:18:05.221680 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe7e667-5d1d-4fa6-bd51-592e611f682f service nova] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:18:05.230545 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1246/2488] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:18:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:18:08.717646 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add883d7-7fa5-4efa-b791-19f226f8f452 service nova] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:18:08.719079 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-add883d7-7fa5-4efa-b791-19f226f8f452 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:18:08.753709 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:18:08.775533 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add883d7-7fa5-4efa-b791-19f226f8f452 service nova] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:18:08.779956 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24f19e-063b-4659-b392-d415b2686327 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:18:08.780574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24f19e-063b-4659-b392-d415b2686327 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:18:08.780858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24f19e-063b-4659-b392-d415b2686327 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:18:08.781137 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24f19e-063b-4659-b392-d415b2686327 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:18:08.782939 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-de24f19e-063b-4659-b392-d415b2686327 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:18:08.794659 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de24f19e-063b-4659-b392-d415b2686327 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:18:08.796223 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de24f19e-063b-4659-b392-d415b2686327 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:18:08.805149 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1243/2489] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:18:08 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:18:14.368727 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7e667-5d1d-4fa6-bd51-592e611f682f service nova] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:18:14.378979 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7e667-5d1d-4fa6-bd51-592e611f682f service nova] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:18:14.384419 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73578b-e754-412a-99c0-2e4d02188d80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:18:14.384419 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73578b-e754-412a-99c0-2e4d02188d80 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:18:14.384419 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73578b-e754-412a-99c0-2e4d02188d80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:18:14.384419 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73578b-e754-412a-99c0-2e4d02188d80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:18:14.385548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-2a73578b-e754-412a-99c0-2e4d02188d80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:18:14.396376 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73578b-e754-412a-99c0-2e4d02188d80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:18:14.397813 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73578b-e754-412a-99c0-2e4d02188d80 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:18:14.406384 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1247/2490] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:18:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:18:18.801033 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=11 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:18:20.174477 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24f19e-063b-4659-b392-d415b2686327 service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:18:20.174477 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-de24f19e-063b-4659-b392-d415b2686327 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:18:20.188829 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24f19e-063b-4659-b392-d415b2686327 service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:18:20.194417 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238a30fa-70a8-41e0-ac1d-e10bcb8539bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:18:20.195072 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a30fa-70a8-41e0-ac1d-e10bcb8539bb service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:18:20.195361 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a30fa-70a8-41e0-ac1d-e10bcb8539bb service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:18:20.195654 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a30fa-70a8-41e0-ac1d-e10bcb8539bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:18:20.197503 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-238a30fa-70a8-41e0-ac1d-e10bcb8539bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:18:20.211106 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238a30fa-70a8-41e0-ac1d-e10bcb8539bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:18:20.212757 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238a30fa-70a8-41e0-ac1d-e10bcb8539bb service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:18:20.224634 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1244/2491] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:18:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:18:24.402651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=12 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:18:30.219463 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=6 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:18:41.410973 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73578b-e754-412a-99c0-2e4d02188d80 service neutron] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:18:41.412458 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-2a73578b-e754-412a-99c0-2e4d02188d80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:18:41.423884 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73578b-e754-412a-99c0-2e4d02188d80 service neutron] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:18:41.427655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a8732-4c45-4987-b605-26b29f127e84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:18:41.428288 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a8732-4c45-4987-b605-26b29f127e84 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:18:41.428575 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a8732-4c45-4987-b605-26b29f127e84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:18:41.428847 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a8732-4c45-4987-b605-26b29f127e84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:18:41.430651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-471a8732-4c45-4987-b605-26b29f127e84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:18:41.494206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471a8732-4c45-4987-b605-26b29f127e84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:18:41.496425 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471a8732-4c45-4987-b605-26b29f127e84 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:18:41.506805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1248/2492] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:18:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:18:45.316759 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238a30fa-70a8-41e0-ac1d-e10bcb8539bb service nova] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:18:45.318200 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-238a30fa-70a8-41e0-ac1d-e10bcb8539bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:18:45.333822 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238a30fa-70a8-41e0-ac1d-e10bcb8539bb service nova] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:18:45.339242 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9504084-9bf5-4e51-ba4a-1c0e0470bade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:18:45.340509 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9504084-9bf5-4e51-ba4a-1c0e0470bade admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:18:45.340790 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9504084-9bf5-4e51-ba4a-1c0e0470bade admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:18:45.341150 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9504084-9bf5-4e51-ba4a-1c0e0470bade admin admin] PATH_INFO: `/v3/users/bb0fe4cb98e0447cad3e126da1a8de9c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:18:45.342517 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9504084-9bf5-4e51-ba4a-1c0e0470bade admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb0fe4cb98e0447cad3e126da1a8de9c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:18:45.345770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9504084-9bf5-4e51-ba4a-1c0e0470bade admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:18:45.534333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b9504084-9bf5-4e51-ba4a-1c0e0470bade admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb0fe4cb98e0447cad3e126da1a8de9c'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:18:45.603292 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b9504084-9bf5-4e51-ba4a-1c0e0470bade admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb0fe4cb98e0447cad3e126da1a8de9c'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:18:45.603953 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b9504084-9bf5-4e51-ba4a-1c0e0470bade admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb0fe4cb98e0447cad3e126da1a8de9c'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:18:45.616765 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b9504084-9bf5-4e51-ba4a-1c0e0470bade admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb0fe4cb98e0447cad3e126da1a8de9c'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:18:45.617537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b9504084-9bf5-4e51-ba4a-1c0e0470bade admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb0fe4cb98e0447cad3e126da1a8de9c'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:18:45.630855 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1245/2493] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:18:45 2022] DELETE /identity/v3/users/bb0fe4cb98e0447cad3e126da1a8de9c => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:18:45.879499 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a8732-4c45-4987-b605-26b29f127e84 service neutron] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:18:45.891678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a8732-4c45-4987-b605-26b29f127e84 service neutron] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:18:45.965909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9cd068-99a5-4136-bd9b-06248cf4d3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:18:45.966559 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9cd068-99a5-4136-bd9b-06248cf4d3ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:18:45.966862 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9cd068-99a5-4136-bd9b-06248cf4d3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:18:45.967161 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9cd068-99a5-4136-bd9b-06248cf4d3ae admin admin] PATH_INFO: `/v3/projects/829c8e80e96d40d682a17185eb4da336` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:18:45.968431 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9cd068-99a5-4136-bd9b-06248cf4d3ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=829c8e80e96d40d682a17185eb4da336)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:18:45.970383 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9cd068-99a5-4136-bd9b-06248cf4d3ae admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:18:46.070747 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-1b9cd068-99a5-4136-bd9b-06248cf4d3ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '829c8e80e96d40d682a17185eb4da336', 'request_id': 'req-1b9cd068-99a5-4136-bd9b-06248cf4d3ae'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:18:46.077494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-1b9cd068-99a5-4136-bd9b-06248cf4d3ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '829c8e80e96d40d682a17185eb4da336', 'request_id': 'req-1b9cd068-99a5-4136-bd9b-06248cf4d3ae'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:18:46.134689 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-1b9cd068-99a5-4136-bd9b-06248cf4d3ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '829c8e80e96d40d682a17185eb4da336', 'request_id': 'req-1b9cd068-99a5-4136-bd9b-06248cf4d3ae'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:18:46.146471 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-1b9cd068-99a5-4136-bd9b-06248cf4d3ae admin admin] The token cache is being invalidate because project 829c8e80e96d40d682a17185eb4da336 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:18:46.147530 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-1b9cd068-99a5-4136-bd9b-06248cf4d3ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:18:46.147530 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-1b9cd068-99a5-4136-bd9b-06248cf4d3ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:18:46.149112 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1249/2494] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:18:45 2022] DELETE /identity/v3/projects/829c8e80e96d40d682a17185eb4da336 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:18:46.154110 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ecd1d-45cc-4e02-a7c7-72a1c0f72c59 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:18:46.154407 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ecd1d-45cc-4e02-a7c7-72a1c0f72c59 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:18:46.154719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ecd1d-45cc-4e02-a7c7-72a1c0f72c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:18:46.176582 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-172ecd1d-45cc-4e02-a7c7-72a1c0f72c59 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:18:46.177819 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-172ecd1d-45cc-4e02-a7c7-72a1c0f72c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:18:46.242449 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1246/2495] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:18:46.250803 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae8a9f-6761-4bc5-b484-614e2122723e None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:18:46.251146 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae8a9f-6761-4bc5-b484-614e2122723e None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:18:46.251406 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae8a9f-6761-4bc5-b484-614e2122723e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:18:46.271157 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-e4ae8a9f-6761-4bc5-b484-614e2122723e None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:18:46.272403 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-e4ae8a9f-6761-4bc5-b484-614e2122723e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:18:46.280936 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1250/2496] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:18:46.285219 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ecd1d-45cc-4e02-a7c7-72a1c0f72c59 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:18:46.295920 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ecd1d-45cc-4e02-a7c7-72a1c0f72c59 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:18:46.300748 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8932b4-55e9-4272-bdf9-6434a53934dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:18:46.301382 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8932b4-55e9-4272-bdf9-6434a53934dc admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:18:46.301792 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8932b4-55e9-4272-bdf9-6434a53934dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:18:46.302080 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8932b4-55e9-4272-bdf9-6434a53934dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:18:46.302786 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8932b4-55e9-4272-bdf9-6434a53934dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:18:46.303128 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8932b4-55e9-4272-bdf9-6434a53934dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:18:46.304753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8932b4-55e9-4272-bdf9-6434a53934dc admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:18:46.318365 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1247/2497] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:18:46.322580 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae8a9f-6761-4bc5-b484-614e2122723e None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:18:46.332074 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae8a9f-6761-4bc5-b484-614e2122723e None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:18:46.337169 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06ed73-13b1-4ff1-a189-ba1404019500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:18:46.337786 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06ed73-13b1-4ff1-a189-ba1404019500 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:18:46.338073 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06ed73-13b1-4ff1-a189-ba1404019500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:18:46.338356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06ed73-13b1-4ff1-a189-ba1404019500 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:18:46.339079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca06ed73-13b1-4ff1-a189-ba1404019500 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:18:46.341512 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca06ed73-13b1-4ff1-a189-ba1404019500 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:18:46.358300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:18:46.419871 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1251/2498] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:18:46 2022] POST /identity/v3/projects => generated 355 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:18:46.423084 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8932b4-55e9-4272-bdf9-6434a53934dc admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:18:46.433973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8932b4-55e9-4272-bdf9-6434a53934dc admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:18:46.456365 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06ed73-13b1-4ff1-a189-ba1404019500 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:18:46.460108 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-ca06ed73-13b1-4ff1-a189-ba1404019500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:18:46.551113 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaad2db-30e2-4349-8510-a08069f03e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:18:46.551893 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaad2db-30e2-4349-8510-a08069f03e48 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:18:46.552349 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaad2db-30e2-4349-8510-a08069f03e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:18:46.552799 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaad2db-30e2-4349-8510-a08069f03e48 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:18:46.553752 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaad2db-30e2-4349-8510-a08069f03e48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:18:46.556267 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaad2db-30e2-4349-8510-a08069f03e48 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:18:46.566344 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:18:46.618458 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06ed73-13b1-4ff1-a189-ba1404019500 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:18:46.627273 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379822f-fa59-4f4c-97e0-210f2dd63d61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:18:46.629133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379822f-fa59-4f4c-97e0-210f2dd63d61 service nova] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:18:46.629133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379822f-fa59-4f4c-97e0-210f2dd63d61 service nova] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:18:46.629133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379822f-fa59-4f4c-97e0-210f2dd63d61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:18:46.630980 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-d379822f-fa59-4f4c-97e0-210f2dd63d61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:18:46.646084 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d379822f-fa59-4f4c-97e0-210f2dd63d61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:18:46.648178 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d379822f-fa59-4f4c-97e0-210f2dd63d61 service nova] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:18:46.660732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1252/2499] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:18:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:18:46.685227 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1248/2500] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:18:46 2022] POST /identity/v3/users => generated 278 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:18:46.689714 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379822f-fa59-4f4c-97e0-210f2dd63d61 service nova] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:18:46.703991 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379822f-fa59-4f4c-97e0-210f2dd63d61 service nova] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:18:46.709873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a48dcb-4f86-46a3-83c3-c2f8bee46fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:18:46.711289 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a48dcb-4f86-46a3-83c3-c2f8bee46fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:18:46.711579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a48dcb-4f86-46a3-83c3-c2f8bee46fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:18:46.711854 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a48dcb-4f86-46a3-83c3-c2f8bee46fc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:18:46.712740 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a48dcb-4f86-46a3-83c3-c2f8bee46fc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:18:46.714967 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a48dcb-4f86-46a3-83c3-c2f8bee46fc3 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:18:46.729065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1253/2501] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:18:46.733403 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaad2db-30e2-4349-8510-a08069f03e48 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:18:46.745221 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaad2db-30e2-4349-8510-a08069f03e48 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:18:46.750392 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655f528-d3aa-4f8b-a977-6ec8b116d92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:18:46.751049 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655f528-d3aa-4f8b-a977-6ec8b116d92a admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:18:46.751340 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655f528-d3aa-4f8b-a977-6ec8b116d92a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:18:46.751629 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655f528-d3aa-4f8b-a977-6ec8b116d92a admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:18:46.752325 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1655f528-d3aa-4f8b-a977-6ec8b116d92a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:18:46.754343 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1655f528-d3aa-4f8b-a977-6ec8b116d92a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:18:46.768324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1249/2502] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:18:46.772097 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a48dcb-4f86-46a3-83c3-c2f8bee46fc3 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:18:46.784449 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a48dcb-4f86-46a3-83c3-c2f8bee46fc3 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:18:46.793360 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a809ba-0353-463d-b59f-31ebce3b39b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:18:46.795381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a809ba-0353-463d-b59f-31ebce3b39b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:18:46.795381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a809ba-0353-463d-b59f-31ebce3b39b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:18:46.795381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a809ba-0353-463d-b59f-31ebce3b39b9 admin admin] PATH_INFO: `/v3/projects/cb9c55c2b1274f2d8fca1f746c78e7f1/users/4ec62c39837d4aac88fe5fa3042617fa/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:18:46.812725 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a809ba-0353-463d-b59f-31ebce3b39b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb9c55c2b1274f2d8fca1f746c78e7f1, user_id=4ec62c39837d4aac88fe5fa3042617fa, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:18:46.820278 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a809ba-0353-463d-b59f-31ebce3b39b9 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:18:46.905834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1254/2503] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:18:46 2022] PUT /identity/v3/projects/cb9c55c2b1274f2d8fca1f746c78e7f1/users/4ec62c39837d4aac88fe5fa3042617fa/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:18:46.921844 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797c670-7812-4032-9758-bf35dfba16a2 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:18:46.922540 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797c670-7812-4032-9758-bf35dfba16a2 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:18:46.923052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797c670-7812-4032-9758-bf35dfba16a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:18:46.960473 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-d797c670-7812-4032-9758-bf35dfba16a2 None None] MFA Rules not processed for user `4ec62c39837d4aac88fe5fa3042617fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:18:46.961629 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d797c670-7812-4032-9758-bf35dfba16a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:18:47.047441 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1250/2504] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:18:46 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:18:47.052413 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a809ba-0353-463d-b59f-31ebce3b39b9 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:18:47.133380 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a809ba-0353-463d-b59f-31ebce3b39b9 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:18:47.139482 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e32e7-f246-46bc-bd3a-3dc8b4273e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:18:47.140235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e32e7-f246-46bc-bd3a-3dc8b4273e27 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:18:47.140710 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e32e7-f246-46bc-bd3a-3dc8b4273e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:18:47.141068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e32e7-f246-46bc-bd3a-3dc8b4273e27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:18:47.141837 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e32e7-f246-46bc-bd3a-3dc8b4273e27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:18:47.143902 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e32e7-f246-46bc-bd3a-3dc8b4273e27 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:18:47.239987 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1255/2505] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:18:47 2022] POST /identity/v3/projects => generated 357 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:18:47.244169 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797c670-7812-4032-9758-bf35dfba16a2 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:18:47.255754 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797c670-7812-4032-9758-bf35dfba16a2 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:18:47.316614 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373a1f59-9d9b-48c9-bd4c-fc75da79a6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:18:47.317259 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a1f59-9d9b-48c9-bd4c-fc75da79a6b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:18:47.317541 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a1f59-9d9b-48c9-bd4c-fc75da79a6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:18:47.317816 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a1f59-9d9b-48c9-bd4c-fc75da79a6b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:18:47.318495 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373a1f59-9d9b-48c9-bd4c-fc75da79a6b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:18:47.320523 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373a1f59-9d9b-48c9-bd4c-fc75da79a6b2 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:18:47.415431 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1251/2506] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:18:47 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:18:47.420048 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e32e7-f246-46bc-bd3a-3dc8b4273e27 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:18:47.430748 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e32e7-f246-46bc-bd3a-3dc8b4273e27 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:18:47.436663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28683a1-2d65-47ff-9533-4ceab230ab77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:18:47.442186 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28683a1-2d65-47ff-9533-4ceab230ab77 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:18:47.442186 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28683a1-2d65-47ff-9533-4ceab230ab77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:18:47.442186 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28683a1-2d65-47ff-9533-4ceab230ab77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:18:47.442186 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28683a1-2d65-47ff-9533-4ceab230ab77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:18:47.442186 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28683a1-2d65-47ff-9533-4ceab230ab77 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:18:47.452305 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1256/2507] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:18:47.456505 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373a1f59-9d9b-48c9-bd4c-fc75da79a6b2 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:18:47.471293 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373a1f59-9d9b-48c9-bd4c-fc75da79a6b2 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:18:47.473825 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ecc4e6-fec4-4c9d-a48f-347e43ab8367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:18:47.474513 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ecc4e6-fec4-4c9d-a48f-347e43ab8367 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:18:47.474801 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ecc4e6-fec4-4c9d-a48f-347e43ab8367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:18:47.475126 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ecc4e6-fec4-4c9d-a48f-347e43ab8367 admin admin] PATH_INFO: `/v3/projects/57ebc843125241d3b8a112029391a2b0/users/df989be201a54680a9339f3d8498268a/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:18:47.491396 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ecc4e6-fec4-4c9d-a48f-347e43ab8367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57ebc843125241d3b8a112029391a2b0, user_id=df989be201a54680a9339f3d8498268a, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:18:47.494110 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ecc4e6-fec4-4c9d-a48f-347e43ab8367 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:18:47.593104 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1252/2508] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:18:47 2022] PUT /identity/v3/projects/57ebc843125241d3b8a112029391a2b0/users/df989be201a54680a9339f3d8498268a/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:18:47.597213 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28683a1-2d65-47ff-9533-4ceab230ab77 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:18:47.608119 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28683a1-2d65-47ff-9533-4ceab230ab77 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:18:47.626315 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:18:47.668124 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ecb60-9d39-4384-9908-7bc2bca017f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:18:47.668839 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ecb60-9d39-4384-9908-7bc2bca017f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:18:47.669204 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ecb60-9d39-4384-9908-7bc2bca017f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:18:47.669556 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ecb60-9d39-4384-9908-7bc2bca017f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:18:47.670356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558ecb60-9d39-4384-9908-7bc2bca017f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:18:47.672347 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558ecb60-9d39-4384-9908-7bc2bca017f7 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:18:47.684699 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1257/2509] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:18:47.690065 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ecc4e6-fec4-4c9d-a48f-347e43ab8367 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:18:47.702441 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ecc4e6-fec4-4c9d-a48f-347e43ab8367 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:18:47.707324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70969512-ccfe-4abe-93dc-03093cd5bcdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:18:47.707995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70969512-ccfe-4abe-93dc-03093cd5bcdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:18:47.708281 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70969512-ccfe-4abe-93dc-03093cd5bcdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:18:47.708590 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70969512-ccfe-4abe-93dc-03093cd5bcdc admin admin] PATH_INFO: `/v3/projects/57ebc843125241d3b8a112029391a2b0/users/df989be201a54680a9339f3d8498268a/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:18:47.710808 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70969512-ccfe-4abe-93dc-03093cd5bcdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=57ebc843125241d3b8a112029391a2b0, user_id=df989be201a54680a9339f3d8498268a, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:18:47.713860 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70969512-ccfe-4abe-93dc-03093cd5bcdc admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:18:47.727835 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1253/2510] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:18:47 2022] PUT /identity/v3/projects/57ebc843125241d3b8a112029391a2b0/users/df989be201a54680a9339f3d8498268a/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:18:47.737752 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9b995-c40b-4740-bc2e-937eb090c235 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:18:47.738045 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9b995-c40b-4740-bc2e-937eb090c235 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:18:47.738325 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9b995-c40b-4740-bc2e-937eb090c235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:18:47.768075 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-63f9b995-c40b-4740-bc2e-937eb090c235 None None] MFA Rules not processed for user `df989be201a54680a9339f3d8498268a`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:18:47.769383 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-63f9b995-c40b-4740-bc2e-937eb090c235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:18:47.856439 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1258/2511] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:18:47 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:18:47.870295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70969512-ccfe-4abe-93dc-03093cd5bcdc admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:18:47.870816 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-70969512-ccfe-4abe-93dc-03093cd5bcdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:18:47.897229 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:18:47.951714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70969512-ccfe-4abe-93dc-03093cd5bcdc admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:18:47.956148 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e3917-dc50-47df-9c19-8bc8a3429ac0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:18:47.957038 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e3917-dc50-47df-9c19-8bc8a3429ac0 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:18:47.958071 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e3917-dc50-47df-9c19-8bc8a3429ac0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:18:47.959007 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e3917-dc50-47df-9c19-8bc8a3429ac0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:18:47.977399 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e3917-dc50-47df-9c19-8bc8a3429ac0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:18:47.977399 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e3917-dc50-47df-9c19-8bc8a3429ac0 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:18:47.988983 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1254/2512] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:18:47 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:18:48.274560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9b995-c40b-4740-bc2e-937eb090c235 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:18:48.276603 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-63f9b995-c40b-4740-bc2e-937eb090c235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:18:48.289641 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9b995-c40b-4740-bc2e-937eb090c235 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:18:48.293951 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f017b69-ec2f-4817-a9e2-991caa140e22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:18:48.294560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f017b69-ec2f-4817-a9e2-991caa140e22 service cinder] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:18:48.294941 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f017b69-ec2f-4817-a9e2-991caa140e22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:18:48.297959 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f017b69-ec2f-4817-a9e2-991caa140e22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:18:48.357200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f017b69-ec2f-4817-a9e2-991caa140e22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:18:48.360058 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f017b69-ec2f-4817-a9e2-991caa140e22 service cinder] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:18:48.373505 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1259/2513] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:18:48 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:18:51.800519 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e3917-dc50-47df-9c19-8bc8a3429ac0 service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:18:51.805192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d07e3917-dc50-47df-9c19-8bc8a3429ac0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:18:51.879423 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e3917-dc50-47df-9c19-8bc8a3429ac0 service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:18:51.884448 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727cf5d1-fcdf-4411-aab6-ec5eb85cd4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:18:51.884862 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727cf5d1-fcdf-4411-aab6-ec5eb85cd4b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:18:51.885140 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727cf5d1-fcdf-4411-aab6-ec5eb85cd4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:18:51.885612 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727cf5d1-fcdf-4411-aab6-ec5eb85cd4b6 admin admin] PATH_INFO: `/v3/users/28d7d91983dc4c0fb218e7bdbc1c6558` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:18:51.887473 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727cf5d1-fcdf-4411-aab6-ec5eb85cd4b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28d7d91983dc4c0fb218e7bdbc1c6558)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:18:51.897208 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727cf5d1-fcdf-4411-aab6-ec5eb85cd4b6 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:18:52.116432 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-727cf5d1-fcdf-4411-aab6-ec5eb85cd4b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28d7d91983dc4c0fb218e7bdbc1c6558'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:18:52.187971 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-727cf5d1-fcdf-4411-aab6-ec5eb85cd4b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28d7d91983dc4c0fb218e7bdbc1c6558'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:18:52.188597 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-727cf5d1-fcdf-4411-aab6-ec5eb85cd4b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28d7d91983dc4c0fb218e7bdbc1c6558'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:18:52.202041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-727cf5d1-fcdf-4411-aab6-ec5eb85cd4b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28d7d91983dc4c0fb218e7bdbc1c6558'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:18:52.202687 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-727cf5d1-fcdf-4411-aab6-ec5eb85cd4b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28d7d91983dc4c0fb218e7bdbc1c6558'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:18:52.216150 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1255/2514] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:18:51 2022] DELETE /identity/v3/users/28d7d91983dc4c0fb218e7bdbc1c6558 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:18:52.462031 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f017b69-ec2f-4817-a9e2-991caa140e22 service cinder] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:18:52.473203 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f017b69-ec2f-4817-a9e2-991caa140e22 service cinder] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:18:52.515255 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:18:52.533874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064628b-2e3e-485e-9212-63fc1569686b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:18:52.534530 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064628b-2e3e-485e-9212-63fc1569686b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:18:52.534814 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064628b-2e3e-485e-9212-63fc1569686b admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:18:52.535137 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064628b-2e3e-485e-9212-63fc1569686b admin admin] PATH_INFO: `/v3/projects/6675985e19c44dabb624420aaa97b0f7` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:18:52.536315 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4064628b-2e3e-485e-9212-63fc1569686b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6675985e19c44dabb624420aaa97b0f7)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:18:52.538467 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4064628b-2e3e-485e-9212-63fc1569686b admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:18:52.680295 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-4064628b-2e3e-485e-9212-63fc1569686b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6675985e19c44dabb624420aaa97b0f7', 'request_id': 'req-4064628b-2e3e-485e-9212-63fc1569686b'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:18:52.687609 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-4064628b-2e3e-485e-9212-63fc1569686b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6675985e19c44dabb624420aaa97b0f7', 'request_id': 'req-4064628b-2e3e-485e-9212-63fc1569686b'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:18:52.762450 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-4064628b-2e3e-485e-9212-63fc1569686b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6675985e19c44dabb624420aaa97b0f7', 'request_id': 'req-4064628b-2e3e-485e-9212-63fc1569686b'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:18:52.773259 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-4064628b-2e3e-485e-9212-63fc1569686b admin admin] The token cache is being invalidate because project 6675985e19c44dabb624420aaa97b0f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:18:52.777863 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-4064628b-2e3e-485e-9212-63fc1569686b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:18:52.777863 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-4064628b-2e3e-485e-9212-63fc1569686b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:18:52.777863 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1260/2515] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:18:52 2022] DELETE /identity/v3/projects/6675985e19c44dabb624420aaa97b0f7 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:18:56.159999 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc311eb-f2d3-47b8-b822-6aeca0abe719 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:18:56.160520 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc311eb-f2d3-47b8-b822-6aeca0abe719 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:18:56.160709 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc311eb-f2d3-47b8-b822-6aeca0abe719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:18:56.182295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-2fc311eb-f2d3-47b8-b822-6aeca0abe719 None None] MFA Rules not processed for user `de853a2dfffb4f05b6f47df2280ae4e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:18:56.184205 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2fc311eb-f2d3-47b8-b822-6aeca0abe719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:18:56.226615 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:18:56.243859 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1256/2516] 10.222.0.40 () {58 vars in 967 bytes} [Wed Aug 3 05:18:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:18:56.412411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064628b-2e3e-485e-9212-63fc1569686b admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:18:56.413895 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-4064628b-2e3e-485e-9212-63fc1569686b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:18:56.486161 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064628b-2e3e-485e-9212-63fc1569686b admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:18:56.491204 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af925141-9b5a-4fe4-b80c-e9639e7121ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1014bcb4d36441aa46042941c9a63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:18:56.491822 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af925141-9b5a-4fe4-b80c-e9639e7121ed service swift] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:18:56.492104 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af925141-9b5a-4fe4-b80c-e9639e7121ed service swift] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:18:56.492382 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af925141-9b5a-4fe4-b80c-e9639e7121ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:18:56.501572 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af925141-9b5a-4fe4-b80c-e9639e7121ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:18:56.503205 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af925141-9b5a-4fe4-b80c-e9639e7121ed service swift] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:18:56.511388 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1261/2517] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 3 05:18:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 05:19:05.003407 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc311eb-f2d3-47b8-b822-6aeca0abe719 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:05.004897 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2fc311eb-f2d3-47b8-b822-6aeca0abe719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:05.079292 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc311eb-f2d3-47b8-b822-6aeca0abe719 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:05.086356 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00ba34-00d0-4d90-8d18-a281c595c63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:05.087040 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00ba34-00d0-4d90-8d18-a281c595c63c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:05.087325 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00ba34-00d0-4d90-8d18-a281c595c63c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:05.087601 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00ba34-00d0-4d90-8d18-a281c595c63c admin admin] PATH_INFO: `/v3/users/79539af2999d48baa11734ced32dccb2` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:05.089091 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00ba34-00d0-4d90-8d18-a281c595c63c admin admin] RBAC: Authorizing `identity:delete_user(user_id=79539af2999d48baa11734ced32dccb2)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:05.091115 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00ba34-00d0-4d90-8d18-a281c595c63c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:05.394075 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-1c00ba34-00d0-4d90-8d18-a281c595c63c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79539af2999d48baa11734ced32dccb2'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:05.467375 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-1c00ba34-00d0-4d90-8d18-a281c595c63c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79539af2999d48baa11734ced32dccb2'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:05.468106 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-1c00ba34-00d0-4d90-8d18-a281c595c63c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79539af2999d48baa11734ced32dccb2'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:05.482932 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-1c00ba34-00d0-4d90-8d18-a281c595c63c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79539af2999d48baa11734ced32dccb2'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:05.483887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-1c00ba34-00d0-4d90-8d18-a281c595c63c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79539af2999d48baa11734ced32dccb2'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:05.497686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1257/2518] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:19:05 2022] DELETE /identity/v3/users/79539af2999d48baa11734ced32dccb2 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:19:05.765805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af925141-9b5a-4fe4-b80c-e9639e7121ed service swift] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:05.779180 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af925141-9b5a-4fe4-b80c-e9639e7121ed service swift] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:05.837889 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f99ca60-3684-49df-9143-571e89ba5c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:05.838527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99ca60-3684-49df-9143-571e89ba5c71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:05.839465 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99ca60-3684-49df-9143-571e89ba5c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:05.839772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99ca60-3684-49df-9143-571e89ba5c71 admin admin] PATH_INFO: `/v3/projects/7d7bf6baa2224b0d910133c5542608e1` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:05.840946 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f99ca60-3684-49df-9143-571e89ba5c71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d7bf6baa2224b0d910133c5542608e1)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:05.843067 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f99ca60-3684-49df-9143-571e89ba5c71 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:05.844498 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:19:05.957747 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-4f99ca60-3684-49df-9143-571e89ba5c71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d7bf6baa2224b0d910133c5542608e1', 'request_id': 'req-4f99ca60-3684-49df-9143-571e89ba5c71'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:05.964957 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-4f99ca60-3684-49df-9143-571e89ba5c71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d7bf6baa2224b0d910133c5542608e1', 'request_id': 'req-4f99ca60-3684-49df-9143-571e89ba5c71'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:06.082131 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-4f99ca60-3684-49df-9143-571e89ba5c71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d7bf6baa2224b0d910133c5542608e1', 'request_id': 'req-4f99ca60-3684-49df-9143-571e89ba5c71'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:06.093910 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-4f99ca60-3684-49df-9143-571e89ba5c71 admin admin] The token cache is being invalidate because project 7d7bf6baa2224b0d910133c5542608e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:19:06.094207 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-4f99ca60-3684-49df-9143-571e89ba5c71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:06.094866 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-4f99ca60-3684-49df-9143-571e89ba5c71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:06.096399 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1262/2519] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:19:05 2022] DELETE /identity/v3/projects/7d7bf6baa2224b0d910133c5542608e1 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:19:06.100483 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00ba34-00d0-4d90-8d18-a281c595c63c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:06.101883 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1c00ba34-00d0-4d90-8d18-a281c595c63c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:06.116024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00ba34-00d0-4d90-8d18-a281c595c63c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:06.180300 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22688fc6-1299-471e-9213-e730d26745f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:06.181026 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22688fc6-1299-471e-9213-e730d26745f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:06.181391 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22688fc6-1299-471e-9213-e730d26745f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:06.181724 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22688fc6-1299-471e-9213-e730d26745f6 admin admin] PATH_INFO: `/v3/users/9f01ab5bffd74ecfa11c9cec030a0dd3` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:06.183100 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22688fc6-1299-471e-9213-e730d26745f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f01ab5bffd74ecfa11c9cec030a0dd3)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:06.185271 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22688fc6-1299-471e-9213-e730d26745f6 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:06.191490 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:19:06.435396 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-22688fc6-1299-471e-9213-e730d26745f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f01ab5bffd74ecfa11c9cec030a0dd3'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:06.545069 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-22688fc6-1299-471e-9213-e730d26745f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f01ab5bffd74ecfa11c9cec030a0dd3'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:06.545729 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-22688fc6-1299-471e-9213-e730d26745f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f01ab5bffd74ecfa11c9cec030a0dd3'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:06.559034 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-22688fc6-1299-471e-9213-e730d26745f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f01ab5bffd74ecfa11c9cec030a0dd3'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:06.559672 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-22688fc6-1299-471e-9213-e730d26745f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f01ab5bffd74ecfa11c9cec030a0dd3'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:06.572020 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1258/2520] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:19:06 2022] DELETE /identity/v3/users/9f01ab5bffd74ecfa11c9cec030a0dd3 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:19:06.804046 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f99ca60-3684-49df-9143-571e89ba5c71 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:06.805728 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-4f99ca60-3684-49df-9143-571e89ba5c71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:06.819778 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f99ca60-3684-49df-9143-571e89ba5c71 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:06.879701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512216a1-6ae4-4d0a-af23-3e31d70f68fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:06.880361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512216a1-6ae4-4d0a-af23-3e31d70f68fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:06.880643 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512216a1-6ae4-4d0a-af23-3e31d70f68fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:06.880917 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512216a1-6ae4-4d0a-af23-3e31d70f68fb admin admin] PATH_INFO: `/v3/projects/f0602692bf964a639462a788ecfd052c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:06.882102 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512216a1-6ae4-4d0a-af23-3e31d70f68fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0602692bf964a639462a788ecfd052c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:06.884248 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512216a1-6ae4-4d0a-af23-3e31d70f68fb admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:06.998579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-512216a1-6ae4-4d0a-af23-3e31d70f68fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0602692bf964a639462a788ecfd052c', 'request_id': 'req-512216a1-6ae4-4d0a-af23-3e31d70f68fb'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:07.004938 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-512216a1-6ae4-4d0a-af23-3e31d70f68fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0602692bf964a639462a788ecfd052c', 'request_id': 'req-512216a1-6ae4-4d0a-af23-3e31d70f68fb'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:07.090162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-512216a1-6ae4-4d0a-af23-3e31d70f68fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0602692bf964a639462a788ecfd052c', 'request_id': 'req-512216a1-6ae4-4d0a-af23-3e31d70f68fb'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:07.104126 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-512216a1-6ae4-4d0a-af23-3e31d70f68fb admin admin] The token cache is being invalidate because project f0602692bf964a639462a788ecfd052c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:19:07.104126 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-512216a1-6ae4-4d0a-af23-3e31d70f68fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:07.104791 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-512216a1-6ae4-4d0a-af23-3e31d70f68fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:07.106876 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1263/2521] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:19:06 2022] DELETE /identity/v3/projects/f0602692bf964a639462a788ecfd052c => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:19:07.646394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22688fc6-1299-471e-9213-e730d26745f6 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:07.647910 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-22688fc6-1299-471e-9213-e730d26745f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:07.661181 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22688fc6-1299-471e-9213-e730d26745f6 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:07.718742 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf7100-22ce-4ffa-95e1-026b629e236d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:07.719511 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf7100-22ce-4ffa-95e1-026b629e236d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:07.719804 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf7100-22ce-4ffa-95e1-026b629e236d admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:07.720134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf7100-22ce-4ffa-95e1-026b629e236d admin admin] PATH_INFO: `/v3/users/4ec62c39837d4aac88fe5fa3042617fa` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:07.721547 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf7100-22ce-4ffa-95e1-026b629e236d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ec62c39837d4aac88fe5fa3042617fa)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:07.723924 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf7100-22ce-4ffa-95e1-026b629e236d admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:07.915230 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0caf7100-22ce-4ffa-95e1-026b629e236d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ec62c39837d4aac88fe5fa3042617fa'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:07.981107 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0caf7100-22ce-4ffa-95e1-026b629e236d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ec62c39837d4aac88fe5fa3042617fa'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:07.981726 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0caf7100-22ce-4ffa-95e1-026b629e236d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ec62c39837d4aac88fe5fa3042617fa'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:07.994213 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0caf7100-22ce-4ffa-95e1-026b629e236d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ec62c39837d4aac88fe5fa3042617fa'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:07.994860 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0caf7100-22ce-4ffa-95e1-026b629e236d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ec62c39837d4aac88fe5fa3042617fa'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:08.007395 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1259/2522] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:19:07 2022] DELETE /identity/v3/users/4ec62c39837d4aac88fe5fa3042617fa => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:19:08.012621 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfefcc2-8a20-4a70-8e65-c7ec58a92e95 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:08.013093 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfefcc2-8a20-4a70-8e65-c7ec58a92e95 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:08.013200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfefcc2-8a20-4a70-8e65-c7ec58a92e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:08.032202 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-8dfefcc2-8a20-4a70-8e65-c7ec58a92e95 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:19:08.033408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-8dfefcc2-8a20-4a70-8e65-c7ec58a92e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:08.046103 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:19:08.093928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1264/2523] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:19:08.103568 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf7100-22ce-4ffa-95e1-026b629e236d admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:08.105013 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0caf7100-22ce-4ffa-95e1-026b629e236d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:08.111536 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:19:08.163896 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf7100-22ce-4ffa-95e1-026b629e236d admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:08.167911 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f695a-9943-4d14-aa06-920994a38c02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:08.168470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f695a-9943-4d14-aa06-920994a38c02 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:08.168751 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f695a-9943-4d14-aa06-920994a38c02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:08.169023 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f695a-9943-4d14-aa06-920994a38c02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:08.179064 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f695a-9943-4d14-aa06-920994a38c02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:08.180515 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f695a-9943-4d14-aa06-920994a38c02 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:08.190736 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1260/2524] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:19:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:19:08.589319 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfefcc2-8a20-4a70-8e65-c7ec58a92e95 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:08.591019 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-8dfefcc2-8a20-4a70-8e65-c7ec58a92e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:08.605366 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfefcc2-8a20-4a70-8e65-c7ec58a92e95 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:08.611306 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9786cb8-5266-4822-8c8a-29be0ede97fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:08.611970 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9786cb8-5266-4822-8c8a-29be0ede97fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:08.612261 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9786cb8-5266-4822-8c8a-29be0ede97fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:08.612963 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9786cb8-5266-4822-8c8a-29be0ede97fb admin admin] PATH_INFO: `/v3/projects/cb9c55c2b1274f2d8fca1f746c78e7f1` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:08.615843 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9786cb8-5266-4822-8c8a-29be0ede97fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb9c55c2b1274f2d8fca1f746c78e7f1)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:08.617776 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9786cb8-5266-4822-8c8a-29be0ede97fb admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:08.703258 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-e9786cb8-5266-4822-8c8a-29be0ede97fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb9c55c2b1274f2d8fca1f746c78e7f1', 'request_id': 'req-e9786cb8-5266-4822-8c8a-29be0ede97fb'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:08.709936 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-e9786cb8-5266-4822-8c8a-29be0ede97fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb9c55c2b1274f2d8fca1f746c78e7f1', 'request_id': 'req-e9786cb8-5266-4822-8c8a-29be0ede97fb'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:08.752941 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-e9786cb8-5266-4822-8c8a-29be0ede97fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb9c55c2b1274f2d8fca1f746c78e7f1', 'request_id': 'req-e9786cb8-5266-4822-8c8a-29be0ede97fb'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:08.765189 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-e9786cb8-5266-4822-8c8a-29be0ede97fb admin admin] The token cache is being invalidate because project cb9c55c2b1274f2d8fca1f746c78e7f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:19:08.765652 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-e9786cb8-5266-4822-8c8a-29be0ede97fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:08.766388 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-e9786cb8-5266-4822-8c8a-29be0ede97fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:08.768461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1265/2525] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:19:08 2022] DELETE /identity/v3/projects/cb9c55c2b1274f2d8fca1f746c78e7f1 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:19:08.772417 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f695a-9943-4d14-aa06-920994a38c02 service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:08.784957 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f695a-9943-4d14-aa06-920994a38c02 service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:08.846876 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e2659-a967-45c5-afa6-d3cc795c653a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:08.847526 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e2659-a967-45c5-afa6-d3cc795c653a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:08.847805 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e2659-a967-45c5-afa6-d3cc795c653a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:08.848080 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e2659-a967-45c5-afa6-d3cc795c653a admin admin] PATH_INFO: `/v3/users/df989be201a54680a9339f3d8498268a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:08.849249 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e2659-a967-45c5-afa6-d3cc795c653a admin admin] RBAC: Authorizing `identity:delete_user(user_id=df989be201a54680a9339f3d8498268a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:08.851244 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e2659-a967-45c5-afa6-d3cc795c653a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:09.004258 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-461e2659-a967-45c5-afa6-d3cc795c653a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df989be201a54680a9339f3d8498268a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:09.087528 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-461e2659-a967-45c5-afa6-d3cc795c653a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df989be201a54680a9339f3d8498268a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:09.088151 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-461e2659-a967-45c5-afa6-d3cc795c653a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df989be201a54680a9339f3d8498268a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:09.101049 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-461e2659-a967-45c5-afa6-d3cc795c653a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df989be201a54680a9339f3d8498268a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:09.101674 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-461e2659-a967-45c5-afa6-d3cc795c653a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df989be201a54680a9339f3d8498268a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:09.114468 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1261/2526] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:19:08 2022] DELETE /identity/v3/users/df989be201a54680a9339f3d8498268a => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:19:09.540894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9786cb8-5266-4822-8c8a-29be0ede97fb admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:09.542577 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-e9786cb8-5266-4822-8c8a-29be0ede97fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:09.562533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9786cb8-5266-4822-8c8a-29be0ede97fb admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:09.625177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6adf7-f727-49a4-8343-1a4fbfd9bfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:09.625820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6adf7-f727-49a4-8343-1a4fbfd9bfc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:09.626111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6adf7-f727-49a4-8343-1a4fbfd9bfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:09.626417 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6adf7-f727-49a4-8343-1a4fbfd9bfc7 admin admin] PATH_INFO: `/v3/projects/57ebc843125241d3b8a112029391a2b0` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:09.627721 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec6adf7-f727-49a4-8343-1a4fbfd9bfc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57ebc843125241d3b8a112029391a2b0)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:09.629638 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec6adf7-f727-49a4-8343-1a4fbfd9bfc7 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:09.737242 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:19:09.739979 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-5ec6adf7-f727-49a4-8343-1a4fbfd9bfc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57ebc843125241d3b8a112029391a2b0', 'request_id': 'req-5ec6adf7-f727-49a4-8343-1a4fbfd9bfc7'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:09.746787 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-5ec6adf7-f727-49a4-8343-1a4fbfd9bfc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57ebc843125241d3b8a112029391a2b0', 'request_id': 'req-5ec6adf7-f727-49a4-8343-1a4fbfd9bfc7'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:09.819083 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-5ec6adf7-f727-49a4-8343-1a4fbfd9bfc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57ebc843125241d3b8a112029391a2b0', 'request_id': 'req-5ec6adf7-f727-49a4-8343-1a4fbfd9bfc7'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:09.830894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-5ec6adf7-f727-49a4-8343-1a4fbfd9bfc7 admin admin] The token cache is being invalidate because project 57ebc843125241d3b8a112029391a2b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:19:09.831241 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-5ec6adf7-f727-49a4-8343-1a4fbfd9bfc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:09.832271 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-5ec6adf7-f727-49a4-8343-1a4fbfd9bfc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:09.834184 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1266/2527] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:19:09 2022] DELETE /identity/v3/projects/57ebc843125241d3b8a112029391a2b0 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:19:09.839124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639ee07-b897-49db-abc0-404642babad9 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:09.841027 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639ee07-b897-49db-abc0-404642babad9 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:09.841027 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639ee07-b897-49db-abc0-404642babad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:09.859198 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-3639ee07-b897-49db-abc0-404642babad9 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:19:09.860387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3639ee07-b897-49db-abc0-404642babad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:09.880849 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:19:09.935244 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1262/2528] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:19:09.944531 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504e2d0-c517-48a3-a4d1-f973ed742415 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:09.944824 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504e2d0-c517-48a3-a4d1-f973ed742415 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:09.945109 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504e2d0-c517-48a3-a4d1-f973ed742415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:09.968537 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-3504e2d0-c517-48a3-a4d1-f973ed742415 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:19:09.969765 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-3504e2d0-c517-48a3-a4d1-f973ed742415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:09.978084 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1267/2529] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:19:09.982888 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639ee07-b897-49db-abc0-404642babad9 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:09.992953 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639ee07-b897-49db-abc0-404642babad9 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:09.998950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182da33-4931-49cd-a872-6567ce79ec45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:09.999654 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182da33-4931-49cd-a872-6567ce79ec45 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:10.000074 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182da33-4931-49cd-a872-6567ce79ec45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:10.000429 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182da33-4931-49cd-a872-6567ce79ec45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:10.001206 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1182da33-4931-49cd-a872-6567ce79ec45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:19:10.001587 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1182da33-4931-49cd-a872-6567ce79ec45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:10.003333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1182da33-4931-49cd-a872-6567ce79ec45 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:10.018752 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1263/2530] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 05:19:10.019847 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504e2d0-c517-48a3-a4d1-f973ed742415 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:10.029266 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504e2d0-c517-48a3-a4d1-f973ed742415 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:10.033977 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5aa75-347c-4d4b-862e-46963f0f1c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:10.034586 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5aa75-347c-4d4b-862e-46963f0f1c8e admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:10.034905 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5aa75-347c-4d4b-862e-46963f0f1c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:10.035186 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5aa75-347c-4d4b-862e-46963f0f1c8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:10.035878 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea5aa75-347c-4d4b-862e-46963f0f1c8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:10.037810 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea5aa75-347c-4d4b-862e-46963f0f1c8e admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:10.127068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1268/2531] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:19:10 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:19:10.136782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182da33-4931-49cd-a872-6567ce79ec45 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:10.148710 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182da33-4931-49cd-a872-6567ce79ec45 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:10.339727 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2450807-11d6-42c2-942c-7e3cbbc22173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:10.340343 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2450807-11d6-42c2-942c-7e3cbbc22173 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:10.340625 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2450807-11d6-42c2-942c-7e3cbbc22173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:10.340901 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2450807-11d6-42c2-942c-7e3cbbc22173 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:10.341569 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2450807-11d6-42c2-942c-7e3cbbc22173 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:10.343507 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2450807-11d6-42c2-942c-7e3cbbc22173 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:10.438189 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1264/2532] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:19:10 2022] POST /identity/v3/users => generated 289 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:19:10.442294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5aa75-347c-4d4b-862e-46963f0f1c8e admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:10.452785 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5aa75-347c-4d4b-862e-46963f0f1c8e admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:10.457434 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3cd21-f19d-46e2-b9a0-8b4638a98284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:10.458040 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3cd21-f19d-46e2-b9a0-8b4638a98284 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:10.458584 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3cd21-f19d-46e2-b9a0-8b4638a98284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:10.458945 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3cd21-f19d-46e2-b9a0-8b4638a98284 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:10.459637 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3cd21-f19d-46e2-b9a0-8b4638a98284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:10.461245 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3cd21-f19d-46e2-b9a0-8b4638a98284 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:10.471669 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1269/2533] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:19:10.475534 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2450807-11d6-42c2-942c-7e3cbbc22173 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:10.486077 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2450807-11d6-42c2-942c-7e3cbbc22173 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:10.490686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68cb724-dcc4-4666-a036-0bcf989a0e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:10.491321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cb724-dcc4-4666-a036-0bcf989a0e59 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:10.491607 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cb724-dcc4-4666-a036-0bcf989a0e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:10.491884 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cb724-dcc4-4666-a036-0bcf989a0e59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:10.492578 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68cb724-dcc4-4666-a036-0bcf989a0e59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:10.494160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68cb724-dcc4-4666-a036-0bcf989a0e59 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:10.504944 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1265/2534] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:19:10.509101 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3cd21-f19d-46e2-b9a0-8b4638a98284 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:10.519474 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3cd21-f19d-46e2-b9a0-8b4638a98284 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:10.524130 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb632ee-fce4-4e17-b53b-6014a0bc5eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:10.524792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb632ee-fce4-4e17-b53b-6014a0bc5eac admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:10.525079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb632ee-fce4-4e17-b53b-6014a0bc5eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:10.525360 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb632ee-fce4-4e17-b53b-6014a0bc5eac admin admin] PATH_INFO: `/v3/projects/b673ffc447ee4438a6d7402bd4aeb3cb/users/40da253036184306a0d582c6157acbbc/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:10.539953 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb632ee-fce4-4e17-b53b-6014a0bc5eac admin admin] RBAC: Authorizing `identity:create_grant(project_id=b673ffc447ee4438a6d7402bd4aeb3cb, user_id=40da253036184306a0d582c6157acbbc, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:10.542661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb632ee-fce4-4e17-b53b-6014a0bc5eac admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:10.615835 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1270/2535] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:19:10 2022] PUT /identity/v3/projects/b673ffc447ee4438a6d7402bd4aeb3cb/users/40da253036184306a0d582c6157acbbc/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:19:10.627110 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17272c35-ac10-404b-8c8b-d27363982e7a None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:10.627506 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17272c35-ac10-404b-8c8b-d27363982e7a None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:10.627909 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17272c35-ac10-404b-8c8b-d27363982e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:10.661881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-17272c35-ac10-404b-8c8b-d27363982e7a None None] MFA Rules not processed for user `40da253036184306a0d582c6157acbbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:19:10.664484 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-17272c35-ac10-404b-8c8b-d27363982e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:10.732049 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1266/2536] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:19:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:19:10.736539 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb632ee-fce4-4e17-b53b-6014a0bc5eac admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:10.798651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb632ee-fce4-4e17-b53b-6014a0bc5eac admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:10.803399 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a822f6f-f9cd-4102-9e22-7607e0eb0dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:10.804012 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a822f6f-f9cd-4102-9e22-7607e0eb0dd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:10.804298 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a822f6f-f9cd-4102-9e22-7607e0eb0dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:10.804574 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a822f6f-f9cd-4102-9e22-7607e0eb0dd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:10.805247 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a822f6f-f9cd-4102-9e22-7607e0eb0dd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:10.807224 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a822f6f-f9cd-4102-9e22-7607e0eb0dd7 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:10.890413 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1271/2537] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:19:10 2022] POST /identity/v3/projects => generated 375 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:19:10.894570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17272c35-ac10-404b-8c8b-d27363982e7a None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:10.906267 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17272c35-ac10-404b-8c8b-d27363982e7a None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:10.925601 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:19:10.966237 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a26035e-c346-433e-96ab-872acf514a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:10.966780 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26035e-c346-433e-96ab-872acf514a17 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:10.967093 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26035e-c346-433e-96ab-872acf514a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:10.967368 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26035e-c346-433e-96ab-872acf514a17 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:10.968052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a26035e-c346-433e-96ab-872acf514a17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:10.969962 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a26035e-c346-433e-96ab-872acf514a17 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:11.073305 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1267/2538] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:19:10 2022] POST /identity/v3/users => generated 294 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:19:11.078007 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a822f6f-f9cd-4102-9e22-7607e0eb0dd7 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:11.088740 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a822f6f-f9cd-4102-9e22-7607e0eb0dd7 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:11.093834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c87b706-d873-4e46-b9bd-fdd334e39bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:11.094466 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87b706-d873-4e46-b9bd-fdd334e39bcc admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:11.094747 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87b706-d873-4e46-b9bd-fdd334e39bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:11.097191 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87b706-d873-4e46-b9bd-fdd334e39bcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:11.097191 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c87b706-d873-4e46-b9bd-fdd334e39bcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:11.097727 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c87b706-d873-4e46-b9bd-fdd334e39bcc admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:11.108910 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1272/2539] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:19:11.113015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a26035e-c346-433e-96ab-872acf514a17 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:11.123742 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a26035e-c346-433e-96ab-872acf514a17 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:11.128317 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efaa44-bcaf-4920-a8ea-ebe3a2f31b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:11.128973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efaa44-bcaf-4920-a8ea-ebe3a2f31b39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:11.129251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efaa44-bcaf-4920-a8ea-ebe3a2f31b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:11.129523 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efaa44-bcaf-4920-a8ea-ebe3a2f31b39 admin admin] PATH_INFO: `/v3/projects/8228e6cf67c749f28e27d4fc02c1a7f0/users/6dc530171703422a9ba83052d229f6fe/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:11.147284 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4efaa44-bcaf-4920-a8ea-ebe3a2f31b39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8228e6cf67c749f28e27d4fc02c1a7f0, user_id=6dc530171703422a9ba83052d229f6fe, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:11.150378 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4efaa44-bcaf-4920-a8ea-ebe3a2f31b39 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:11.267734 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1268/2540] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:19:11 2022] PUT /identity/v3/projects/8228e6cf67c749f28e27d4fc02c1a7f0/users/6dc530171703422a9ba83052d229f6fe/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:19:11.271790 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c87b706-d873-4e46-b9bd-fdd334e39bcc admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:11.282394 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c87b706-d873-4e46-b9bd-fdd334e39bcc admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:11.333877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:19:11.352502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5206d8-e891-4905-8e38-7125c0eb6b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:11.352911 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5206d8-e891-4905-8e38-7125c0eb6b59 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:11.353261 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5206d8-e891-4905-8e38-7125c0eb6b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:11.353639 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5206d8-e891-4905-8e38-7125c0eb6b59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:11.354586 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5206d8-e891-4905-8e38-7125c0eb6b59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:11.357296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5206d8-e891-4905-8e38-7125c0eb6b59 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:11.369999 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1273/2541] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:19:11.372906 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efaa44-bcaf-4920-a8ea-ebe3a2f31b39 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:11.385084 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efaa44-bcaf-4920-a8ea-ebe3a2f31b39 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:11.398748 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f42a7d-2ea9-434b-8e3f-edae822331f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:11.398748 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f42a7d-2ea9-434b-8e3f-edae822331f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:11.398748 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f42a7d-2ea9-434b-8e3f-edae822331f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:11.398748 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f42a7d-2ea9-434b-8e3f-edae822331f9 admin admin] PATH_INFO: `/v3/projects/8228e6cf67c749f28e27d4fc02c1a7f0/users/6dc530171703422a9ba83052d229f6fe/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:11.399125 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f42a7d-2ea9-434b-8e3f-edae822331f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8228e6cf67c749f28e27d4fc02c1a7f0, user_id=6dc530171703422a9ba83052d229f6fe, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:11.400934 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f42a7d-2ea9-434b-8e3f-edae822331f9 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:11.412757 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1269/2542] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:19:11 2022] PUT /identity/v3/projects/8228e6cf67c749f28e27d4fc02c1a7f0/users/6dc530171703422a9ba83052d229f6fe/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:19:11.423067 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8cac1-2c11-46de-a550-802eb292c75b None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:11.423382 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8cac1-2c11-46de-a550-802eb292c75b None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:11.423682 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8cac1-2c11-46de-a550-802eb292c75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:11.458546 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-e0d8cac1-2c11-46de-a550-802eb292c75b None None] MFA Rules not processed for user `6dc530171703422a9ba83052d229f6fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:19:11.459832 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-e0d8cac1-2c11-46de-a550-802eb292c75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:11.528677 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1274/2543] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:19:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:19:11.538976 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f42a7d-2ea9-434b-8e3f-edae822331f9 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:11.540650 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a5f42a7d-2ea9-434b-8e3f-edae822331f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:11.604444 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f42a7d-2ea9-434b-8e3f-edae822331f9 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:11.608119 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b3042-cb75-4c3c-996a-a6dff67da0f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:11.608727 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b3042-cb75-4c3c-996a-a6dff67da0f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:11.609007 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b3042-cb75-4c3c-996a-a6dff67da0f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:11.609279 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b3042-cb75-4c3c-996a-a6dff67da0f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:11.673534 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1b3042-cb75-4c3c-996a-a6dff67da0f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:11.675039 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1b3042-cb75-4c3c-996a-a6dff67da0f2 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:11.684833 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1270/2544] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:19:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:19:14.336675 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172ad97-92e1-4e95-9a5b-90ed05280572 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:14.336675 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172ad97-92e1-4e95-9a5b-90ed05280572 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:14.336675 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172ad97-92e1-4e95-9a5b-90ed05280572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:14.355978 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-1172ad97-92e1-4e95-9a5b-90ed05280572 None None] MFA Rules not processed for user `de853a2dfffb4f05b6f47df2280ae4e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:19:14.357272 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-1172ad97-92e1-4e95-9a5b-90ed05280572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:14.415594 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1275/2545] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Aug 3 05:19:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:19:14.428033 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b3042-cb75-4c3c-996a-a6dff67da0f2 service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:14.431126 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-bd1b3042-cb75-4c3c-996a-a6dff67da0f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:14.462721 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:19:14.500453 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b3042-cb75-4c3c-996a-a6dff67da0f2 service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:14.506297 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3f6c0e-513e-4adc-8e00-5c70e16813d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1014bcb4d36441aa46042941c9a63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:14.506944 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f6c0e-513e-4adc-8e00-5c70e16813d2 service swift] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:14.507230 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f6c0e-513e-4adc-8e00-5c70e16813d2 service swift] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:14.507508 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f6c0e-513e-4adc-8e00-5c70e16813d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:14.516205 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3f6c0e-513e-4adc-8e00-5c70e16813d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:14.517802 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3f6c0e-513e-4adc-8e00-5c70e16813d2 service swift] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:14.525788 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1271/2546] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 3 05:19:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 05:19:15.447865 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172ad97-92e1-4e95-9a5b-90ed05280572 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:15.449652 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-1172ad97-92e1-4e95-9a5b-90ed05280572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:15.558371 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172ad97-92e1-4e95-9a5b-90ed05280572 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:15.567309 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9ebc5-0faf-4a61-a59b-0ada43d741db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:15.567959 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9ebc5-0faf-4a61-a59b-0ada43d741db service nova] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:15.568245 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9ebc5-0faf-4a61-a59b-0ada43d741db service nova] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:15.568523 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9ebc5-0faf-4a61-a59b-0ada43d741db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:15.570372 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-cdd9ebc5-0faf-4a61-a59b-0ada43d741db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:15.582105 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:19:15.585904 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9ebc5-0faf-4a61-a59b-0ada43d741db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:15.588571 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9ebc5-0faf-4a61-a59b-0ada43d741db service nova] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:15.602502 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1276/2547] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:19:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:19:18.171563 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3f6c0e-513e-4adc-8e00-5c70e16813d2 service swift] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:18.173857 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ce3f6c0e-513e-4adc-8e00-5c70e16813d2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:18.188089 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3f6c0e-513e-4adc-8e00-5c70e16813d2 service swift] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:18.192003 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cad111-cd55-4b06-b403-311b3f4fdd55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:18.192617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cad111-cd55-4b06-b403-311b3f4fdd55 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:18.192896 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cad111-cd55-4b06-b403-311b3f4fdd55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:18.193174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cad111-cd55-4b06-b403-311b3f4fdd55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:18.204552 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cad111-cd55-4b06-b403-311b3f4fdd55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:18.205993 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cad111-cd55-4b06-b403-311b3f4fdd55 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:18.221200 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1272/2548] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:19:18 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:19:24.410436 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9ebc5-0faf-4a61-a59b-0ada43d741db service nova] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:24.412894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-cdd9ebc5-0faf-4a61-a59b-0ada43d741db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:24.507554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9ebc5-0faf-4a61-a59b-0ada43d741db service nova] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:24.513784 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af7631-b4cb-46a6-a4e1-39585b8e5867 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59846d88f4314c2382aa5cb8bae5053b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:24.514501 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af7631-b4cb-46a6-a4e1-39585b8e5867 service placement] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:24.514885 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af7631-b4cb-46a6-a4e1-39585b8e5867 service placement] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:24.515242 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af7631-b4cb-46a6-a4e1-39585b8e5867 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:24.517072 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-d2af7631-b4cb-46a6-a4e1-39585b8e5867 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:24.529955 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af7631-b4cb-46a6-a4e1-39585b8e5867 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:24.531601 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af7631-b4cb-46a6-a4e1-39585b8e5867 service placement] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:24.542061 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1277/2549] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:19:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:19:28.214450 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=25 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:19:33.983315 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cad111-cd55-4b06-b403-311b3f4fdd55 service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:33.984797 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-67cad111-cd55-4b06-b403-311b3f4fdd55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:34.044175 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cad111-cd55-4b06-b403-311b3f4fdd55 service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:34.047788 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170c4dd-8e9a-4053-ac0d-931c002f6917 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:34.048398 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170c4dd-8e9a-4053-ac0d-931c002f6917 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:34.048683 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170c4dd-8e9a-4053-ac0d-931c002f6917 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:34.048962 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170c4dd-8e9a-4053-ac0d-931c002f6917 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:34.050691 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d170c4dd-8e9a-4053-ac0d-931c002f6917 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:34.060072 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d170c4dd-8e9a-4053-ac0d-931c002f6917 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:34.061486 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d170c4dd-8e9a-4053-ac0d-931c002f6917 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:34.069408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1273/2550] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:19:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:19:34.435103 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189c09b-8ea7-41c4-8707-082955603d1e None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:34.437178 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189c09b-8ea7-41c4-8707-082955603d1e None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:34.437178 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189c09b-8ea7-41c4-8707-082955603d1e None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:34.437542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1278/2551] 10.222.0.40 () {54 vars in 921 bytes} [Wed Aug 3 05:19:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 05:19:34.444208 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03c7a5-31a0-4856-af02-911bd1ecc788 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:34.444494 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03c7a5-31a0-4856-af02-911bd1ecc788 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:34.444770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03c7a5-31a0-4856-af02-911bd1ecc788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:34.468644 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-5a03c7a5-31a0-4856-af02-911bd1ecc788 None None] MFA Rules not processed for user `a3fa4a6c16274c18849869b55b33ac62`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:19:34.469843 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-5a03c7a5-31a0-4856-af02-911bd1ecc788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:34.489087 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1274/2552] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Aug 3 05:19:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:19:34.493178 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189c09b-8ea7-41c4-8707-082955603d1e None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:34.494909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-b189c09b-8ea7-41c4-8707-082955603d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:34.564598 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189c09b-8ea7-41c4-8707-082955603d1e None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:34.568223 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f0e8a-0918-4e99-b941-993d2711b214 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cc14b33895443b893fab487bfe1858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:34.568837 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f0e8a-0918-4e99-b941-993d2711b214 service glance] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:34.569120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f0e8a-0918-4e99-b941-993d2711b214 service glance] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:34.569393 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f0e8a-0918-4e99-b941-993d2711b214 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:34.580105 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f0e8a-0918-4e99-b941-993d2711b214 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:34.581544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f0e8a-0918-4e99-b941-993d2711b214 service glance] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:34.592497 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1279/2553] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:19:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:19:34.780762 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8896040-f293-46a0-9b56-72d88d6c855c None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:34.780762 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8896040-f293-46a0-9b56-72d88d6c855c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:34.780762 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8896040-f293-46a0-9b56-72d88d6c855c None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:34.780762 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1275/2554] 10.222.0.40 () {54 vars in 921 bytes} [Wed Aug 3 05:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 05:19:34.783875 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8744fe6-55bc-48e2-a080-7eb19440d7a4 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:34.784228 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8744fe6-55bc-48e2-a080-7eb19440d7a4 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:34.784493 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8744fe6-55bc-48e2-a080-7eb19440d7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:34.805421 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-b8744fe6-55bc-48e2-a080-7eb19440d7a4 None None] MFA Rules not processed for user `cff15ea4cbc64b068aa5f6586b5bff87`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:19:34.806661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-b8744fe6-55bc-48e2-a080-7eb19440d7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:34.814806 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1280/2555] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Aug 3 05:19:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:19:34.827689 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8896040-f293-46a0-9b56-72d88d6c855c None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:34.829676 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c8896040-f293-46a0-9b56-72d88d6c855c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:34.844205 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8896040-f293-46a0-9b56-72d88d6c855c None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:34.847796 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0926e54-4713-49b9-a94b-86f4072190ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:34.848397 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0926e54-4713-49b9-a94b-86f4072190ee service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:34.848675 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0926e54-4713-49b9-a94b-86f4072190ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:34.848947 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0926e54-4713-49b9-a94b-86f4072190ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:34.856713 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0926e54-4713-49b9-a94b-86f4072190ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:34.858168 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0926e54-4713-49b9-a94b-86f4072190ee service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:34.865993 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1276/2556] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:19:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:19:34.947842 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5644fa0-c659-4416-bf59-5ad60f57979a None None] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:34.948307 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5644fa0-c659-4416-bf59-5ad60f57979a None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:34.948587 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5644fa0-c659-4416-bf59-5ad60f57979a None None] PATH_INFO: `/` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:34.950509 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1281/2557] 10.222.0.40 () {54 vars in 921 bytes} [Wed Aug 3 05:19:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 05:19:34.956319 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b0126-3185-457f-bc2a-a7c7e3b08296 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:34.956612 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b0126-3185-457f-bc2a-a7c7e3b08296 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:34.956893 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b0126-3185-457f-bc2a-a7c7e3b08296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:34.978281 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-880b0126-3185-457f-bc2a-a7c7e3b08296 None None] MFA Rules not processed for user `a3fa4a6c16274c18849869b55b33ac62`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:19:34.979586 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-880b0126-3185-457f-bc2a-a7c7e3b08296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:34.989447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1277/2558] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Aug 3 05:19:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:19:34.999971 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5644fa0-c659-4416-bf59-5ad60f57979a None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:35.011287 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5644fa0-c659-4416-bf59-5ad60f57979a None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:35.014949 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4007cf-436d-49bb-b564-3c526158def6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:35.015560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4007cf-436d-49bb-b564-3c526158def6 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:35.015843 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4007cf-436d-49bb-b564-3c526158def6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:35.016122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4007cf-436d-49bb-b564-3c526158def6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:35.027953 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4007cf-436d-49bb-b564-3c526158def6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:35.029794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4007cf-436d-49bb-b564-3c526158def6 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:35.041336 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1282/2559] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:19:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:19:41.684372 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8b1c0-c3c1-4a65-8586-a08791a8b6b3 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:41.684914 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8b1c0-c3c1-4a65-8586-a08791a8b6b3 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:41.684914 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8b1c0-c3c1-4a65-8586-a08791a8b6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:41.704009 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-7cd8b1c0-c3c1-4a65-8586-a08791a8b6b3 None None] MFA Rules not processed for user `de853a2dfffb4f05b6f47df2280ae4e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:19:41.705226 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7cd8b1c0-c3c1-4a65-8586-a08791a8b6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:41.713069 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1278/2560] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Aug 3 05:19:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:19:41.727764 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4007cf-436d-49bb-b564-3c526158def6 service neutron] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:41.748381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4007cf-436d-49bb-b564-3c526158def6 service neutron] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:41.755547 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe46e9-7a65-4c2f-ae87-553029f90691 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1014bcb4d36441aa46042941c9a63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:41.755547 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe46e9-7a65-4c2f-ae87-553029f90691 service swift] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:41.755547 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe46e9-7a65-4c2f-ae87-553029f90691 service swift] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:41.755547 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe46e9-7a65-4c2f-ae87-553029f90691 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:41.766959 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afe46e9-7a65-4c2f-ae87-553029f90691 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:41.768753 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afe46e9-7a65-4c2f-ae87-553029f90691 service swift] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:41.778889 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1283/2561] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 3 05:19:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 03 05:19:46.595716 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8b1c0-c3c1-4a65-8586-a08791a8b6b3 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:46.703436 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8b1c0-c3c1-4a65-8586-a08791a8b6b3 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:46.708580 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b727ee-cd83-42de-bc71-93c4866e9e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:46.709235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b727ee-cd83-42de-bc71-93c4866e9e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:46.709521 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b727ee-cd83-42de-bc71-93c4866e9e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:46.709803 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b727ee-cd83-42de-bc71-93c4866e9e68 admin admin] PATH_INFO: `/v3/users/40da253036184306a0d582c6157acbbc` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:46.711025 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b727ee-cd83-42de-bc71-93c4866e9e68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40da253036184306a0d582c6157acbbc)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:46.712957 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b727ee-cd83-42de-bc71-93c4866e9e68 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:46.894671 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e9b727ee-cd83-42de-bc71-93c4866e9e68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40da253036184306a0d582c6157acbbc'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:46.986959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e9b727ee-cd83-42de-bc71-93c4866e9e68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40da253036184306a0d582c6157acbbc'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:46.987695 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e9b727ee-cd83-42de-bc71-93c4866e9e68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40da253036184306a0d582c6157acbbc'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:47.038536 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e9b727ee-cd83-42de-bc71-93c4866e9e68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40da253036184306a0d582c6157acbbc'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:47.040842 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e9b727ee-cd83-42de-bc71-93c4866e9e68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40da253036184306a0d582c6157acbbc'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:47.042608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:19:47.069937 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1279/2562] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:19:46 2022] DELETE /identity/v3/users/40da253036184306a0d582c6157acbbc => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:19:47.074419 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df1ac65-889f-40d8-8878-182925fb6a1c None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:47.075290 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df1ac65-889f-40d8-8878-182925fb6a1c None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:47.075722 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df1ac65-889f-40d8-8878-182925fb6a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:47.084959 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=76 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:19:47.099199 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-1df1ac65-889f-40d8-8878-182925fb6a1c None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:19:47.100777 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-1df1ac65-889f-40d8-8878-182925fb6a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:47.183043 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1284/2563] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:19:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:19:47.193039 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b727ee-cd83-42de-bc71-93c4866e9e68 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:47.194521 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e9b727ee-cd83-42de-bc71-93c4866e9e68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:47.271426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b727ee-cd83-42de-bc71-93c4866e9e68 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:47.276441 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c439652d-33f2-4298-a3f8-a65f48576da0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:47.277090 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439652d-33f2-4298-a3f8-a65f48576da0 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:47.277367 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439652d-33f2-4298-a3f8-a65f48576da0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:47.277645 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439652d-33f2-4298-a3f8-a65f48576da0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:47.290665 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c439652d-33f2-4298-a3f8-a65f48576da0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:47.292306 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c439652d-33f2-4298-a3f8-a65f48576da0 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:47.303189 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1280/2564] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:19:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:19:48.044208 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df1ac65-889f-40d8-8878-182925fb6a1c None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:48.047461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-1df1ac65-889f-40d8-8878-182925fb6a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:48.061937 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df1ac65-889f-40d8-8878-182925fb6a1c None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:48.067514 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f0aa7e-de29-49c6-8c45-61e774e3621a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:48.068296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0aa7e-de29-49c6-8c45-61e774e3621a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:48.068579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0aa7e-de29-49c6-8c45-61e774e3621a admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:48.069080 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0aa7e-de29-49c6-8c45-61e774e3621a admin admin] PATH_INFO: `/v3/projects/b673ffc447ee4438a6d7402bd4aeb3cb` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:48.070596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f0aa7e-de29-49c6-8c45-61e774e3621a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b673ffc447ee4438a6d7402bd4aeb3cb)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:48.073362 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f0aa7e-de29-49c6-8c45-61e774e3621a admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:48.228046 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-24f0aa7e-de29-49c6-8c45-61e774e3621a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b673ffc447ee4438a6d7402bd4aeb3cb', 'request_id': 'req-24f0aa7e-de29-49c6-8c45-61e774e3621a'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:48.238318 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-24f0aa7e-de29-49c6-8c45-61e774e3621a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b673ffc447ee4438a6d7402bd4aeb3cb', 'request_id': 'req-24f0aa7e-de29-49c6-8c45-61e774e3621a'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:48.338177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-24f0aa7e-de29-49c6-8c45-61e774e3621a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b673ffc447ee4438a6d7402bd4aeb3cb', 'request_id': 'req-24f0aa7e-de29-49c6-8c45-61e774e3621a'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:48.351714 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-24f0aa7e-de29-49c6-8c45-61e774e3621a admin admin] The token cache is being invalidate because project b673ffc447ee4438a6d7402bd4aeb3cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:19:48.352373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-24f0aa7e-de29-49c6-8c45-61e774e3621a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:48.353278 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-24f0aa7e-de29-49c6-8c45-61e774e3621a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:48.355628 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1285/2565] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:19:48 2022] DELETE /identity/v3/projects/b673ffc447ee4438a6d7402bd4aeb3cb => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:19:48.360436 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c439652d-33f2-4298-a3f8-a65f48576da0 service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:48.373863 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c439652d-33f2-4298-a3f8-a65f48576da0 service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:48.436249 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b55c4-cf63-4c43-b2b9-1b7c5edee273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:48.436895 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b55c4-cf63-4c43-b2b9-1b7c5edee273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:48.437174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b55c4-cf63-4c43-b2b9-1b7c5edee273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:48.437449 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b55c4-cf63-4c43-b2b9-1b7c5edee273 admin admin] PATH_INFO: `/v3/users/6dc530171703422a9ba83052d229f6fe` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:48.438623 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b55c4-cf63-4c43-b2b9-1b7c5edee273 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dc530171703422a9ba83052d229f6fe)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:48.440587 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b55c4-cf63-4c43-b2b9-1b7c5edee273 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:48.737109 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-6e4b55c4-cf63-4c43-b2b9-1b7c5edee273 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dc530171703422a9ba83052d229f6fe'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:48.961117 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-6e4b55c4-cf63-4c43-b2b9-1b7c5edee273 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dc530171703422a9ba83052d229f6fe'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:48.961777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-6e4b55c4-cf63-4c43-b2b9-1b7c5edee273 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dc530171703422a9ba83052d229f6fe'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:48.976487 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-6e4b55c4-cf63-4c43-b2b9-1b7c5edee273 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dc530171703422a9ba83052d229f6fe'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:48.977208 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-6e4b55c4-cf63-4c43-b2b9-1b7c5edee273 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dc530171703422a9ba83052d229f6fe'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:48.990121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1281/2566] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:19:48 2022] DELETE /identity/v3/users/6dc530171703422a9ba83052d229f6fe => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:19:49.527259 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f0aa7e-de29-49c6-8c45-61e774e3621a admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:49.527259 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-24f0aa7e-de29-49c6-8c45-61e774e3621a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:49.543765 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f0aa7e-de29-49c6-8c45-61e774e3621a admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:49.629786 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84938c5c-ad58-4d13-8c95-c17baf578b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:49.629786 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84938c5c-ad58-4d13-8c95-c17baf578b66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:49.629786 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84938c5c-ad58-4d13-8c95-c17baf578b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:49.629786 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84938c5c-ad58-4d13-8c95-c17baf578b66 admin admin] PATH_INFO: `/v3/projects/8228e6cf67c749f28e27d4fc02c1a7f0` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:49.630557 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84938c5c-ad58-4d13-8c95-c17baf578b66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8228e6cf67c749f28e27d4fc02c1a7f0)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:49.630557 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84938c5c-ad58-4d13-8c95-c17baf578b66 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:49.920719 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:19:49.942194 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-84938c5c-ad58-4d13-8c95-c17baf578b66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8228e6cf67c749f28e27d4fc02c1a7f0', 'request_id': 'req-84938c5c-ad58-4d13-8c95-c17baf578b66'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:49.949405 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-84938c5c-ad58-4d13-8c95-c17baf578b66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8228e6cf67c749f28e27d4fc02c1a7f0', 'request_id': 'req-84938c5c-ad58-4d13-8c95-c17baf578b66'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:50.036719 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-84938c5c-ad58-4d13-8c95-c17baf578b66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8228e6cf67c749f28e27d4fc02c1a7f0', 'request_id': 'req-84938c5c-ad58-4d13-8c95-c17baf578b66'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:50.048940 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-84938c5c-ad58-4d13-8c95-c17baf578b66 admin admin] The token cache is being invalidate because project 8228e6cf67c749f28e27d4fc02c1a7f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:19:50.049230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-84938c5c-ad58-4d13-8c95-c17baf578b66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:50.049894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-84938c5c-ad58-4d13-8c95-c17baf578b66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:50.051591 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1286/2567] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:19:49 2022] DELETE /identity/v3/projects/8228e6cf67c749f28e27d4fc02c1a7f0 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:19:50.057358 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e50c1-1cd8-4889-a3d5-0cf8995581f2 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:50.057980 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e50c1-1cd8-4889-a3d5-0cf8995581f2 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:50.057980 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e50c1-1cd8-4889-a3d5-0cf8995581f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:50.077616 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-672e50c1-1cd8-4889-a3d5-0cf8995581f2 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:19:50.081786 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-672e50c1-1cd8-4889-a3d5-0cf8995581f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:50.084519 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:19:50.139511 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1282/2568] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:19:50.147905 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ff912-a9ca-42b4-9e7c-310b07932733 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:50.148131 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ff912-a9ca-42b4-9e7c-310b07932733 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:50.148370 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ff912-a9ca-42b4-9e7c-310b07932733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:50.168599 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-355ff912-a9ca-42b4-9e7c-310b07932733 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:19:50.171662 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-355ff912-a9ca-42b4-9e7c-310b07932733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:50.178355 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1287/2569] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:19:50.182801 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e50c1-1cd8-4889-a3d5-0cf8995581f2 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:50.192627 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e50c1-1cd8-4889-a3d5-0cf8995581f2 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:50.198414 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5dc56-584c-4bf8-bd62-7dca167de081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:50.199177 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5dc56-584c-4bf8-bd62-7dca167de081 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:50.199592 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5dc56-584c-4bf8-bd62-7dca167de081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:50.200004 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5dc56-584c-4bf8-bd62-7dca167de081 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:50.200859 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be5dc56-584c-4bf8-bd62-7dca167de081 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:19:50.201239 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be5dc56-584c-4bf8-bd62-7dca167de081 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:50.203597 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be5dc56-584c-4bf8-bd62-7dca167de081 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:50.217822 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1283/2570] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 05:19:50.222452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ff912-a9ca-42b4-9e7c-310b07932733 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:50.232338 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ff912-a9ca-42b4-9e7c-310b07932733 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:50.237597 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aff0d7-e010-47cc-a7a9-81af6ce32ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:50.238215 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aff0d7-e010-47cc-a7a9-81af6ce32ea7 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:50.238499 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aff0d7-e010-47cc-a7a9-81af6ce32ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:50.238775 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aff0d7-e010-47cc-a7a9-81af6ce32ea7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:50.239469 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aff0d7-e010-47cc-a7a9-81af6ce32ea7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:50.241398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aff0d7-e010-47cc-a7a9-81af6ce32ea7 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:50.352036 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1288/2571] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:19:50 2022] POST /identity/v3/projects => generated 361 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:19:50.358139 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5dc56-584c-4bf8-bd62-7dca167de081 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:50.368347 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5dc56-584c-4bf8-bd62-7dca167de081 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:50.425027 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081bed0d-1d07-435f-97d6-a9bda7bed475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:50.425564 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081bed0d-1d07-435f-97d6-a9bda7bed475 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:50.425851 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081bed0d-1d07-435f-97d6-a9bda7bed475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:50.426129 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081bed0d-1d07-435f-97d6-a9bda7bed475 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:50.426821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081bed0d-1d07-435f-97d6-a9bda7bed475 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:50.429359 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081bed0d-1d07-435f-97d6-a9bda7bed475 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:50.508973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1284/2572] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:19:50 2022] POST /identity/v3/users => generated 281 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:19:50.513111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aff0d7-e010-47cc-a7a9-81af6ce32ea7 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:50.524150 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aff0d7-e010-47cc-a7a9-81af6ce32ea7 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:50.528965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f57a4-2e5c-499b-8a91-d69e7b3d5aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:50.530834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f57a4-2e5c-499b-8a91-d69e7b3d5aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:50.530834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f57a4-2e5c-499b-8a91-d69e7b3d5aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:50.530834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f57a4-2e5c-499b-8a91-d69e7b3d5aa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:50.531428 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694f57a4-2e5c-499b-8a91-d69e7b3d5aa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:50.533051 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694f57a4-2e5c-499b-8a91-d69e7b3d5aa8 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:50.543292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1289/2573] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:19:50.547211 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081bed0d-1d07-435f-97d6-a9bda7bed475 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:50.557346 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081bed0d-1d07-435f-97d6-a9bda7bed475 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:50.562111 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f26e0f-47c2-49b1-947c-8bc1067a8933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:50.562618 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f26e0f-47c2-49b1-947c-8bc1067a8933 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:50.562925 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f26e0f-47c2-49b1-947c-8bc1067a8933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:50.563204 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f26e0f-47c2-49b1-947c-8bc1067a8933 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:50.563896 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f26e0f-47c2-49b1-947c-8bc1067a8933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:50.565485 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f26e0f-47c2-49b1-947c-8bc1067a8933 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:50.575998 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1285/2574] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:19:50.580435 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f57a4-2e5c-499b-8a91-d69e7b3d5aa8 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:50.591523 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f57a4-2e5c-499b-8a91-d69e7b3d5aa8 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:50.597032 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bfc945-23a3-463a-82cc-6199d1bf1455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:50.597697 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bfc945-23a3-463a-82cc-6199d1bf1455 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:50.597983 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bfc945-23a3-463a-82cc-6199d1bf1455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:50.598288 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bfc945-23a3-463a-82cc-6199d1bf1455 admin admin] PATH_INFO: `/v3/projects/10201cb68a144139b0da9dc3ee099c60/users/f3b2345e11fd495e8f855e7b8c69bd02/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:50.613621 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bfc945-23a3-463a-82cc-6199d1bf1455 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10201cb68a144139b0da9dc3ee099c60, user_id=f3b2345e11fd495e8f855e7b8c69bd02, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:50.616403 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bfc945-23a3-463a-82cc-6199d1bf1455 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:50.698376 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1290/2575] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:19:50 2022] PUT /identity/v3/projects/10201cb68a144139b0da9dc3ee099c60/users/f3b2345e11fd495e8f855e7b8c69bd02/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:19:50.709386 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeeb8ba-ed48-4a0b-8dae-933e98368495 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:50.709721 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeeb8ba-ed48-4a0b-8dae-933e98368495 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:50.709968 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeeb8ba-ed48-4a0b-8dae-933e98368495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:50.740957 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-edeeb8ba-ed48-4a0b-8dae-933e98368495 None None] MFA Rules not processed for user `f3b2345e11fd495e8f855e7b8c69bd02`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:19:50.742202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-edeeb8ba-ed48-4a0b-8dae-933e98368495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:50.810537 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1286/2576] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:19:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:19:50.846865 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bfc945-23a3-463a-82cc-6199d1bf1455 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:50.848655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-78bfc945-23a3-463a-82cc-6199d1bf1455 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:50.912149 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bfc945-23a3-463a-82cc-6199d1bf1455 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:50.916128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393bf96-6bcb-43f4-b730-920f4e7bbcc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:50.918252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393bf96-6bcb-43f4-b730-920f4e7bbcc1 service cinder] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:50.918252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393bf96-6bcb-43f4-b730-920f4e7bbcc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:50.918252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393bf96-6bcb-43f4-b730-920f4e7bbcc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:50.926410 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3393bf96-6bcb-43f4-b730-920f4e7bbcc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:50.927893 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3393bf96-6bcb-43f4-b730-920f4e7bbcc1 service cinder] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:50.937995 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1291/2577] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:19:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:19:50.955124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeeb8ba-ed48-4a0b-8dae-933e98368495 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:50.964740 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:19:51.021316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeeb8ba-ed48-4a0b-8dae-933e98368495 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:51.025900 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71ff5b-0ee2-4787-be93-bdea6c3a14a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:51.026539 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71ff5b-0ee2-4787-be93-bdea6c3a14a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:51.026823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71ff5b-0ee2-4787-be93-bdea6c3a14a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:51.027131 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71ff5b-0ee2-4787-be93-bdea6c3a14a2 admin admin] PATH_INFO: `/v3/users/f3b2345e11fd495e8f855e7b8c69bd02` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:51.028311 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c71ff5b-0ee2-4787-be93-bdea6c3a14a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3b2345e11fd495e8f855e7b8c69bd02)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:51.030231 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c71ff5b-0ee2-4787-be93-bdea6c3a14a2 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:51.245625 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7c71ff5b-0ee2-4787-be93-bdea6c3a14a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3b2345e11fd495e8f855e7b8c69bd02'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:51.372392 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7c71ff5b-0ee2-4787-be93-bdea6c3a14a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3b2345e11fd495e8f855e7b8c69bd02'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:51.373095 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7c71ff5b-0ee2-4787-be93-bdea6c3a14a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3b2345e11fd495e8f855e7b8c69bd02'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:51.389052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7c71ff5b-0ee2-4787-be93-bdea6c3a14a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3b2345e11fd495e8f855e7b8c69bd02'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:51.391424 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7c71ff5b-0ee2-4787-be93-bdea6c3a14a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3b2345e11fd495e8f855e7b8c69bd02'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:51.405684 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1287/2578] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:19:50 2022] DELETE /identity/v3/users/f3b2345e11fd495e8f855e7b8c69bd02 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:19:51.411165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa93b1f-6bfd-412e-bd57-4f86e2e7a65b None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:51.414877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa93b1f-6bfd-412e-bd57-4f86e2e7a65b None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:51.414877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa93b1f-6bfd-412e-bd57-4f86e2e7a65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:51.442747 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-cfa93b1f-6bfd-412e-bd57-4f86e2e7a65b None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:19:51.444136 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-cfa93b1f-6bfd-412e-bd57-4f86e2e7a65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:51.515050 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:19:51.523430 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1292/2579] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:19:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:19:51.534629 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71ff5b-0ee2-4787-be93-bdea6c3a14a2 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:51.536059 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7c71ff5b-0ee2-4787-be93-bdea6c3a14a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:51.594050 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71ff5b-0ee2-4787-be93-bdea6c3a14a2 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:51.598263 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a028b4-9b88-429b-a3df-a16253b5c3b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:51.598968 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a028b4-9b88-429b-a3df-a16253b5c3b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:51.599253 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a028b4-9b88-429b-a3df-a16253b5c3b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:51.599529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a028b4-9b88-429b-a3df-a16253b5c3b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:51.610003 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a028b4-9b88-429b-a3df-a16253b5c3b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:51.611542 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a028b4-9b88-429b-a3df-a16253b5c3b6 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:51.621331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1288/2580] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:19:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:19:52.102210 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa93b1f-6bfd-412e-bd57-4f86e2e7a65b None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:52.103718 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-cfa93b1f-6bfd-412e-bd57-4f86e2e7a65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:52.115857 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa93b1f-6bfd-412e-bd57-4f86e2e7a65b None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:52.120552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08e03a-9217-44af-8c56-edc5240b571c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:52.121212 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08e03a-9217-44af-8c56-edc5240b571c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:52.121522 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08e03a-9217-44af-8c56-edc5240b571c admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:52.122117 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08e03a-9217-44af-8c56-edc5240b571c admin admin] PATH_INFO: `/v3/projects/10201cb68a144139b0da9dc3ee099c60` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:52.123560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08e03a-9217-44af-8c56-edc5240b571c admin admin] RBAC: Authorizing `identity:delete_project(project_id=10201cb68a144139b0da9dc3ee099c60)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:52.126892 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08e03a-9217-44af-8c56-edc5240b571c admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:52.232653 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-be08e03a-9217-44af-8c56-edc5240b571c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10201cb68a144139b0da9dc3ee099c60', 'request_id': 'req-be08e03a-9217-44af-8c56-edc5240b571c'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:52.239147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-be08e03a-9217-44af-8c56-edc5240b571c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10201cb68a144139b0da9dc3ee099c60', 'request_id': 'req-be08e03a-9217-44af-8c56-edc5240b571c'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:52.299445 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-be08e03a-9217-44af-8c56-edc5240b571c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10201cb68a144139b0da9dc3ee099c60', 'request_id': 'req-be08e03a-9217-44af-8c56-edc5240b571c'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:52.314044 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-be08e03a-9217-44af-8c56-edc5240b571c admin admin] The token cache is being invalidate because project 10201cb68a144139b0da9dc3ee099c60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:19:52.314768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-be08e03a-9217-44af-8c56-edc5240b571c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:52.315625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-be08e03a-9217-44af-8c56-edc5240b571c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:52.317275 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1293/2581] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:19:52 2022] DELETE /identity/v3/projects/10201cb68a144139b0da9dc3ee099c60 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:19:52.322372 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de378be-16ed-41bc-be34-fa7ad5e5966e None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:52.322901 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de378be-16ed-41bc-be34-fa7ad5e5966e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:52.323014 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de378be-16ed-41bc-be34-fa7ad5e5966e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:52.343534 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-7de378be-16ed-41bc-be34-fa7ad5e5966e None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:19:52.344768 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7de378be-16ed-41bc-be34-fa7ad5e5966e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:52.371568 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:19:52.416838 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1289/2582] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:19:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:19:52.425487 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25503d-2d24-4a1e-ab57-39efec71a9fa None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:52.425773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25503d-2d24-4a1e-ab57-39efec71a9fa None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:52.426050 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25503d-2d24-4a1e-ab57-39efec71a9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:52.448809 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-4a25503d-2d24-4a1e-ab57-39efec71a9fa None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:19:52.452855 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-4a25503d-2d24-4a1e-ab57-39efec71a9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:52.459356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1294/2583] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:19:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:19:52.463792 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de378be-16ed-41bc-be34-fa7ad5e5966e None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:52.474209 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de378be-16ed-41bc-be34-fa7ad5e5966e None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:52.478947 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95939868-7a2c-4fb4-be62-55de0793f7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:52.481314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95939868-7a2c-4fb4-be62-55de0793f7a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:52.481314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95939868-7a2c-4fb4-be62-55de0793f7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:52.481314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95939868-7a2c-4fb4-be62-55de0793f7a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:52.481314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95939868-7a2c-4fb4-be62-55de0793f7a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:19:52.481314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95939868-7a2c-4fb4-be62-55de0793f7a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:52.482809 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95939868-7a2c-4fb4-be62-55de0793f7a4 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:52.495035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1290/2584] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:19:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:19:52.500278 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a25503d-2d24-4a1e-ab57-39efec71a9fa None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:52.509788 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a25503d-2d24-4a1e-ab57-39efec71a9fa None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:52.514680 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6598d2a-ce4e-47fd-8e34-833bbeb73dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:52.515323 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6598d2a-ce4e-47fd-8e34-833bbeb73dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:52.515608 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6598d2a-ce4e-47fd-8e34-833bbeb73dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:52.515885 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6598d2a-ce4e-47fd-8e34-833bbeb73dd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:52.516557 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6598d2a-ce4e-47fd-8e34-833bbeb73dd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:52.518715 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6598d2a-ce4e-47fd-8e34-833bbeb73dd0 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:52.601768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1295/2585] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:19:52 2022] POST /identity/v3/projects => generated 375 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:19:52.605973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95939868-7a2c-4fb4-be62-55de0793f7a4 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:52.617617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95939868-7a2c-4fb4-be62-55de0793f7a4 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:52.676055 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dee16e-7851-4512-b159-db4dc8369956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:52.676732 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dee16e-7851-4512-b159-db4dc8369956 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:52.677128 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dee16e-7851-4512-b159-db4dc8369956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:52.677282 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dee16e-7851-4512-b159-db4dc8369956 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:52.677967 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dee16e-7851-4512-b159-db4dc8369956 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:52.679941 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dee16e-7851-4512-b159-db4dc8369956 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:52.772789 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1291/2586] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:19:52 2022] POST /identity/v3/users => generated 288 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:19:52.776860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6598d2a-ce4e-47fd-8e34-833bbeb73dd0 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:52.787566 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6598d2a-ce4e-47fd-8e34-833bbeb73dd0 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:52.792525 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6f188-c0a3-4d67-b9f9-eca5a536d208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:52.793161 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6f188-c0a3-4d67-b9f9-eca5a536d208 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:52.793427 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6f188-c0a3-4d67-b9f9-eca5a536d208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:52.793699 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6f188-c0a3-4d67-b9f9-eca5a536d208 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:52.794391 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6f188-c0a3-4d67-b9f9-eca5a536d208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:52.796571 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6f188-c0a3-4d67-b9f9-eca5a536d208 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:52.807319 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1296/2587] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:19:52.813119 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dee16e-7851-4512-b159-db4dc8369956 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:52.823239 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dee16e-7851-4512-b159-db4dc8369956 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:52.827859 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea41b0c-ef84-4569-8ec7-24015c321308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:52.828464 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea41b0c-ef84-4569-8ec7-24015c321308 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:52.828833 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea41b0c-ef84-4569-8ec7-24015c321308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:52.829021 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea41b0c-ef84-4569-8ec7-24015c321308 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:52.829712 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea41b0c-ef84-4569-8ec7-24015c321308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:52.831590 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea41b0c-ef84-4569-8ec7-24015c321308 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:52.842481 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1292/2588] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:19:52.847795 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6f188-c0a3-4d67-b9f9-eca5a536d208 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:52.858040 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6f188-c0a3-4d67-b9f9-eca5a536d208 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:52.863370 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041dce3-e81a-4f55-ab1f-13095d2da433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:52.864023 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041dce3-e81a-4f55-ab1f-13095d2da433 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:52.864305 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041dce3-e81a-4f55-ab1f-13095d2da433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:52.864578 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041dce3-e81a-4f55-ab1f-13095d2da433 admin admin] PATH_INFO: `/v3/projects/f85f83913b8f46e4a27b1b7d4b3f4323/users/cfe11280e823434fa22247a9787a8da7/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:52.879408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041dce3-e81a-4f55-ab1f-13095d2da433 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f85f83913b8f46e4a27b1b7d4b3f4323, user_id=cfe11280e823434fa22247a9787a8da7, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:52.882065 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041dce3-e81a-4f55-ab1f-13095d2da433 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:52.987084 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1297/2589] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:19:52 2022] PUT /identity/v3/projects/f85f83913b8f46e4a27b1b7d4b3f4323/users/cfe11280e823434fa22247a9787a8da7/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:19:52.997141 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29f3c5-b4b6-46c2-9df3-1f77bd06bd52 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:52.997528 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29f3c5-b4b6-46c2-9df3-1f77bd06bd52 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:52.997734 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29f3c5-b4b6-46c2-9df3-1f77bd06bd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:53.028003 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-ea29f3c5-b4b6-46c2-9df3-1f77bd06bd52 None None] MFA Rules not processed for user `cfe11280e823434fa22247a9787a8da7`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:19:53.029222 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ea29f3c5-b4b6-46c2-9df3-1f77bd06bd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:53.102551 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1293/2590] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:19:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:19:53.107218 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041dce3-e81a-4f55-ab1f-13095d2da433 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:53.173068 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041dce3-e81a-4f55-ab1f-13095d2da433 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:53.177763 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731c028-6986-4630-ad41-89b7f7f9c744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:53.178432 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731c028-6986-4630-ad41-89b7f7f9c744 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:53.178886 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731c028-6986-4630-ad41-89b7f7f9c744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:53.179170 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731c028-6986-4630-ad41-89b7f7f9c744 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:53.179723 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731c028-6986-4630-ad41-89b7f7f9c744 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:53.181648 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731c028-6986-4630-ad41-89b7f7f9c744 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:53.288663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:19:53.299043 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1298/2591] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:19:53 2022] POST /identity/v3/projects => generated 375 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:19:53.301320 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29f3c5-b4b6-46c2-9df3-1f77bd06bd52 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:53.312767 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29f3c5-b4b6-46c2-9df3-1f77bd06bd52 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:53.333442 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:19:53.373228 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd8d055-94ea-465b-b1ac-05a23fdbabb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:53.374679 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8d055-94ea-465b-b1ac-05a23fdbabb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:53.375204 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8d055-94ea-465b-b1ac-05a23fdbabb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:53.376024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8d055-94ea-465b-b1ac-05a23fdbabb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:53.376496 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd8d055-94ea-465b-b1ac-05a23fdbabb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:53.378452 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd8d055-94ea-465b-b1ac-05a23fdbabb0 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:53.453147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1294/2592] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:19:53 2022] POST /identity/v3/users => generated 294 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:19:53.454319 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731c028-6986-4630-ad41-89b7f7f9c744 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:53.470186 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731c028-6986-4630-ad41-89b7f7f9c744 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:53.476686 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1169d05d-97a5-4e03-bdf4-fde182cfcd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:53.477427 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1169d05d-97a5-4e03-bdf4-fde182cfcd22 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:53.477778 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1169d05d-97a5-4e03-bdf4-fde182cfcd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:53.478130 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1169d05d-97a5-4e03-bdf4-fde182cfcd22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:53.479258 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1169d05d-97a5-4e03-bdf4-fde182cfcd22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:53.482496 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1169d05d-97a5-4e03-bdf4-fde182cfcd22 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:53.497526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1299/2593] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:19:53.505508 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd8d055-94ea-465b-b1ac-05a23fdbabb0 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:53.516637 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd8d055-94ea-465b-b1ac-05a23fdbabb0 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:53.522928 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf04b7-324f-4d53-846e-26f49d82377c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:53.523627 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf04b7-324f-4d53-846e-26f49d82377c admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:53.523915 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf04b7-324f-4d53-846e-26f49d82377c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:53.524198 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf04b7-324f-4d53-846e-26f49d82377c admin admin] PATH_INFO: `/v3/projects/2b6ade103aee49059379437c796e93fb/users/4cd4d05903d744aba3a4cb71862672b9/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:53.543885 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf04b7-324f-4d53-846e-26f49d82377c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b6ade103aee49059379437c796e93fb, user_id=4cd4d05903d744aba3a4cb71862672b9, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:53.547735 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf04b7-324f-4d53-846e-26f49d82377c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:53.655890 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1295/2594] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:19:53 2022] PUT /identity/v3/projects/2b6ade103aee49059379437c796e93fb/users/4cd4d05903d744aba3a4cb71862672b9/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:19:53.656071 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1169d05d-97a5-4e03-bdf4-fde182cfcd22 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:53.671760 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1169d05d-97a5-4e03-bdf4-fde182cfcd22 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:53.733167 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22f20f-0e6b-4be6-9927-27f72f998288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:53.734331 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22f20f-0e6b-4be6-9927-27f72f998288 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:53.734619 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22f20f-0e6b-4be6-9927-27f72f998288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:53.734941 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22f20f-0e6b-4be6-9927-27f72f998288 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:53.735644 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22f20f-0e6b-4be6-9927-27f72f998288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:53.737271 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22f20f-0e6b-4be6-9927-27f72f998288 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:53.747361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1300/2595] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:19:53.751600 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf04b7-324f-4d53-846e-26f49d82377c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:53.761978 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf04b7-324f-4d53-846e-26f49d82377c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:53.767071 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d7232-51a5-4617-b9a1-ee7e847b83bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:53.767736 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d7232-51a5-4617-b9a1-ee7e847b83bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:53.768023 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d7232-51a5-4617-b9a1-ee7e847b83bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:53.768307 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d7232-51a5-4617-b9a1-ee7e847b83bc admin admin] PATH_INFO: `/v3/projects/2b6ade103aee49059379437c796e93fb/users/4cd4d05903d744aba3a4cb71862672b9/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:53.770276 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d7232-51a5-4617-b9a1-ee7e847b83bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b6ade103aee49059379437c796e93fb, user_id=4cd4d05903d744aba3a4cb71862672b9, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:53.772994 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d7232-51a5-4617-b9a1-ee7e847b83bc admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:53.783917 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1296/2596] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:19:53 2022] PUT /identity/v3/projects/2b6ade103aee49059379437c796e93fb/users/4cd4d05903d744aba3a4cb71862672b9/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:19:53.794044 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07176be-4dbb-4eb3-adbe-36412c9e8acd None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:53.794367 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07176be-4dbb-4eb3-adbe-36412c9e8acd None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:53.794652 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07176be-4dbb-4eb3-adbe-36412c9e8acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:53.822468 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-c07176be-4dbb-4eb3-adbe-36412c9e8acd None None] MFA Rules not processed for user `4cd4d05903d744aba3a4cb71862672b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:19:53.823730 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-c07176be-4dbb-4eb3-adbe-36412c9e8acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:53.909598 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1301/2597] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:19:53 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:19:53.918525 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d7232-51a5-4617-b9a1-ee7e847b83bc admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:53.919692 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ff7d7232-51a5-4617-b9a1-ee7e847b83bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:53.983805 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d7232-51a5-4617-b9a1-ee7e847b83bc admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:53.987738 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6dfd7a-29e5-4eeb-8640-ac7ec444a412 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:53.988394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6dfd7a-29e5-4eeb-8640-ac7ec444a412 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:53.988679 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6dfd7a-29e5-4eeb-8640-ac7ec444a412 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:53.988955 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6dfd7a-29e5-4eeb-8640-ac7ec444a412 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:54.047469 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6dfd7a-29e5-4eeb-8640-ac7ec444a412 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:54.048915 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6dfd7a-29e5-4eeb-8640-ac7ec444a412 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:54.057909 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1297/2598] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:19:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:19:55.099977 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22439bb6-c49c-44d8-8c80-4d5b967e4b46 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:55.100473 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22439bb6-c49c-44d8-8c80-4d5b967e4b46 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:55.100570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22439bb6-c49c-44d8-8c80-4d5b967e4b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:55.123125 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-22439bb6-c49c-44d8-8c80-4d5b967e4b46 None None] MFA Rules not processed for user `de853a2dfffb4f05b6f47df2280ae4e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:19:55.124612 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-22439bb6-c49c-44d8-8c80-4d5b967e4b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:55.186561 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1302/2599] 10.222.0.40 () {58 vars in 967 bytes} [Wed Aug 3 05:19:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:19:55.199975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6dfd7a-29e5-4eeb-8640-ac7ec444a412 service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:55.201711 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9b6dfd7a-29e5-4eeb-8640-ac7ec444a412 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:55.241001 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:19:55.279758 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6dfd7a-29e5-4eeb-8640-ac7ec444a412 service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:55.285592 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75327939-ed9f-4df2-88dd-12a3ad08e96c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1014bcb4d36441aa46042941c9a63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:55.286216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75327939-ed9f-4df2-88dd-12a3ad08e96c service swift] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:55.286502 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75327939-ed9f-4df2-88dd-12a3ad08e96c service swift] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:55.286782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75327939-ed9f-4df2-88dd-12a3ad08e96c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:55.296402 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75327939-ed9f-4df2-88dd-12a3ad08e96c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:55.298477 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75327939-ed9f-4df2-88dd-12a3ad08e96c service swift] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:55.306852 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1298/2600] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Aug 3 05:19:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 05:19:58.111651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22439bb6-c49c-44d8-8c80-4d5b967e4b46 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:58.113223 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-22439bb6-c49c-44d8-8c80-4d5b967e4b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:58.131845 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:19:58.169638 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22439bb6-c49c-44d8-8c80-4d5b967e4b46 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:58.173266 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7739b69-9b34-4beb-9264-73c676164fc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:58.173884 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7739b69-9b34-4beb-9264-73c676164fc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:58.174170 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7739b69-9b34-4beb-9264-73c676164fc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:58.174448 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7739b69-9b34-4beb-9264-73c676164fc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:58.176219 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-c7739b69-9b34-4beb-9264-73c676164fc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:58.205047 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75327939-ed9f-4df2-88dd-12a3ad08e96c service swift] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:58.243093 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7739b69-9b34-4beb-9264-73c676164fc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:58.244590 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7739b69-9b34-4beb-9264-73c676164fc9 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:58.255415 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1303/2601] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:19:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:19:58.269682 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75327939-ed9f-4df2-88dd-12a3ad08e96c service swift] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:58.274213 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e0911-9c77-4f84-8f92-8a38ac32fb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:58.274864 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e0911-9c77-4f84-8f92-8a38ac32fb43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:58.275173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e0911-9c77-4f84-8f92-8a38ac32fb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:58.275451 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e0911-9c77-4f84-8f92-8a38ac32fb43 admin admin] PATH_INFO: `/v3/users/cfe11280e823434fa22247a9787a8da7` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:58.276611 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568e0911-9c77-4f84-8f92-8a38ac32fb43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfe11280e823434fa22247a9787a8da7)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:58.278540 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568e0911-9c77-4f84-8f92-8a38ac32fb43 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:58.485971 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-568e0911-9c77-4f84-8f92-8a38ac32fb43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfe11280e823434fa22247a9787a8da7'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:58.550911 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-568e0911-9c77-4f84-8f92-8a38ac32fb43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfe11280e823434fa22247a9787a8da7'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:58.551581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-568e0911-9c77-4f84-8f92-8a38ac32fb43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfe11280e823434fa22247a9787a8da7'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:58.566141 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-568e0911-9c77-4f84-8f92-8a38ac32fb43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfe11280e823434fa22247a9787a8da7'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:58.566927 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-568e0911-9c77-4f84-8f92-8a38ac32fb43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfe11280e823434fa22247a9787a8da7'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:58.579990 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1299/2602] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:19:58 2022] DELETE /identity/v3/users/cfe11280e823434fa22247a9787a8da7 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:19:58.584818 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920be67-b13a-4c4f-b5dd-1e3526bb1ea0 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:58.585112 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920be67-b13a-4c4f-b5dd-1e3526bb1ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:58.585394 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920be67-b13a-4c4f-b5dd-1e3526bb1ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:58.605253 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-7920be67-b13a-4c4f-b5dd-1e3526bb1ea0 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:19:58.606501 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-7920be67-b13a-4c4f-b5dd-1e3526bb1ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:58.678198 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1304/2603] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:19:58.685026 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e0911-9c77-4f84-8f92-8a38ac32fb43 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:58.686457 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-568e0911-9c77-4f84-8f92-8a38ac32fb43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:58.734318 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:19:58.748476 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e0911-9c77-4f84-8f92-8a38ac32fb43 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:58.753023 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979babf8-5241-4ec0-9335-bf89e0a52c47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:58.753717 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979babf8-5241-4ec0-9335-bf89e0a52c47 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:58.754005 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979babf8-5241-4ec0-9335-bf89e0a52c47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:58.754286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979babf8-5241-4ec0-9335-bf89e0a52c47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:58.764738 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979babf8-5241-4ec0-9335-bf89e0a52c47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:58.766202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979babf8-5241-4ec0-9335-bf89e0a52c47 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:58.776855 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1300/2604] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:19:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:19:59.215024 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920be67-b13a-4c4f-b5dd-1e3526bb1ea0 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:59.216444 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-7920be67-b13a-4c4f-b5dd-1e3526bb1ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:19:59.234944 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920be67-b13a-4c4f-b5dd-1e3526bb1ea0 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:59.239669 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be4d8e-2fcf-4f3e-ab39-699528eda127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:59.240394 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be4d8e-2fcf-4f3e-ab39-699528eda127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:59.240853 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be4d8e-2fcf-4f3e-ab39-699528eda127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:59.241288 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be4d8e-2fcf-4f3e-ab39-699528eda127 admin admin] PATH_INFO: `/v3/projects/f85f83913b8f46e4a27b1b7d4b3f4323` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:59.242658 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be4d8e-2fcf-4f3e-ab39-699528eda127 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f85f83913b8f46e4a27b1b7d4b3f4323)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:59.249850 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be4d8e-2fcf-4f3e-ab39-699528eda127 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:59.366651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-00be4d8e-2fcf-4f3e-ab39-699528eda127 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f85f83913b8f46e4a27b1b7d4b3f4323', 'request_id': 'req-00be4d8e-2fcf-4f3e-ab39-699528eda127'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:59.373166 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-00be4d8e-2fcf-4f3e-ab39-699528eda127 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f85f83913b8f46e4a27b1b7d4b3f4323', 'request_id': 'req-00be4d8e-2fcf-4f3e-ab39-699528eda127'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:59.446717 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-00be4d8e-2fcf-4f3e-ab39-699528eda127 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f85f83913b8f46e4a27b1b7d4b3f4323', 'request_id': 'req-00be4d8e-2fcf-4f3e-ab39-699528eda127'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:59.448300 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:19:59.464342 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-00be4d8e-2fcf-4f3e-ab39-699528eda127 admin admin] The token cache is being invalidate because project f85f83913b8f46e4a27b1b7d4b3f4323 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:19:59.464640 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-00be4d8e-2fcf-4f3e-ab39-699528eda127 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:59.465493 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-00be4d8e-2fcf-4f3e-ab39-699528eda127 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:59.466930 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1305/2605] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:19:59 2022] DELETE /identity/v3/projects/f85f83913b8f46e4a27b1b7d4b3f4323 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:19:59.471732 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979babf8-5241-4ec0-9335-bf89e0a52c47 service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:19:59.483010 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979babf8-5241-4ec0-9335-bf89e0a52c47 service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:19:59.548362 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aed112-3a75-4ede-8ad5-2b690e7569b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:19:59.549137 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aed112-3a75-4ede-8ad5-2b690e7569b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:19:59.549803 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aed112-3a75-4ede-8ad5-2b690e7569b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:19:59.549803 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aed112-3a75-4ede-8ad5-2b690e7569b9 admin admin] PATH_INFO: `/v3/users/4cd4d05903d744aba3a4cb71862672b9` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:19:59.551594 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aed112-3a75-4ede-8ad5-2b690e7569b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cd4d05903d744aba3a4cb71862672b9)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:19:59.553128 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aed112-3a75-4ede-8ad5-2b690e7569b9 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:19:59.742082 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-55aed112-3a75-4ede-8ad5-2b690e7569b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cd4d05903d744aba3a4cb71862672b9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:59.817526 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-55aed112-3a75-4ede-8ad5-2b690e7569b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cd4d05903d744aba3a4cb71862672b9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:59.818261 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-55aed112-3a75-4ede-8ad5-2b690e7569b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cd4d05903d744aba3a4cb71862672b9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:59.832413 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-55aed112-3a75-4ede-8ad5-2b690e7569b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cd4d05903d744aba3a4cb71862672b9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:59.833186 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-55aed112-3a75-4ede-8ad5-2b690e7569b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cd4d05903d744aba3a4cb71862672b9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:19:59.846656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1301/2606] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:19:59 2022] DELETE /identity/v3/users/4cd4d05903d744aba3a4cb71862672b9 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:20:00.264913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be4d8e-2fcf-4f3e-ab39-699528eda127 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:00.266863 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-00be4d8e-2fcf-4f3e-ab39-699528eda127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:00.291150 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be4d8e-2fcf-4f3e-ab39-699528eda127 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:00.363516 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902736fb-7c6a-40cf-9405-5a81b7c4f0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:00.364175 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902736fb-7c6a-40cf-9405-5a81b7c4f0f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:00.364663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902736fb-7c6a-40cf-9405-5a81b7c4f0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:00.364936 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902736fb-7c6a-40cf-9405-5a81b7c4f0f5 admin admin] PATH_INFO: `/v3/projects/2b6ade103aee49059379437c796e93fb` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:00.366397 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902736fb-7c6a-40cf-9405-5a81b7c4f0f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b6ade103aee49059379437c796e93fb)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:00.368397 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902736fb-7c6a-40cf-9405-5a81b7c4f0f5 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:00.474574 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-902736fb-7c6a-40cf-9405-5a81b7c4f0f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b6ade103aee49059379437c796e93fb', 'request_id': 'req-902736fb-7c6a-40cf-9405-5a81b7c4f0f5'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:00.481960 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-902736fb-7c6a-40cf-9405-5a81b7c4f0f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b6ade103aee49059379437c796e93fb', 'request_id': 'req-902736fb-7c6a-40cf-9405-5a81b7c4f0f5'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:00.555029 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-902736fb-7c6a-40cf-9405-5a81b7c4f0f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b6ade103aee49059379437c796e93fb', 'request_id': 'req-902736fb-7c6a-40cf-9405-5a81b7c4f0f5'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:00.567169 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-902736fb-7c6a-40cf-9405-5a81b7c4f0f5 admin admin] The token cache is being invalidate because project 2b6ade103aee49059379437c796e93fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:20:00.567556 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-902736fb-7c6a-40cf-9405-5a81b7c4f0f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:00.568275 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-902736fb-7c6a-40cf-9405-5a81b7c4f0f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:00.569903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1306/2607] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:20:00 2022] DELETE /identity/v3/projects/2b6ade103aee49059379437c796e93fb => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:20:00.574748 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b3868-2c8e-4039-b0ed-5e81bd5d4df2 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:00.575284 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b3868-2c8e-4039-b0ed-5e81bd5d4df2 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:00.575392 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b3868-2c8e-4039-b0ed-5e81bd5d4df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:00.596837 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-9f0b3868-2c8e-4039-b0ed-5e81bd5d4df2 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:00.601264 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9f0b3868-2c8e-4039-b0ed-5e81bd5d4df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:00.679804 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1302/2608] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:20:00.734637 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113077a-c7dd-479e-9725-d27d16048dc7 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:00.734874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113077a-c7dd-479e-9725-d27d16048dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:00.735182 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113077a-c7dd-479e-9725-d27d16048dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:00.754761 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-3113077a-c7dd-479e-9725-d27d16048dc7 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:00.756161 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-3113077a-c7dd-479e-9725-d27d16048dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:00.764296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1307/2609] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:20:00.769079 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b3868-2c8e-4039-b0ed-5e81bd5d4df2 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:00.773915 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:00.779163 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b3868-2c8e-4039-b0ed-5e81bd5d4df2 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:00.784604 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f32b668-87b7-4040-9a48-cb8763e3f5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:00.790460 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32b668-87b7-4040-9a48-cb8763e3f5cd admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:00.790460 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32b668-87b7-4040-9a48-cb8763e3f5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:00.790460 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32b668-87b7-4040-9a48-cb8763e3f5cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:00.790460 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f32b668-87b7-4040-9a48-cb8763e3f5cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:20:00.790460 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f32b668-87b7-4040-9a48-cb8763e3f5cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:00.790460 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f32b668-87b7-4040-9a48-cb8763e3f5cd admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:00.800608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1303/2610] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:20:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:20:00.805197 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113077a-c7dd-479e-9725-d27d16048dc7 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:00.815080 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113077a-c7dd-479e-9725-d27d16048dc7 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:00.820152 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f36ebc-271e-489a-a26f-a8cb8ec0518b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:00.820777 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f36ebc-271e-489a-a26f-a8cb8ec0518b admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:00.821062 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f36ebc-271e-489a-a26f-a8cb8ec0518b admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:00.821341 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f36ebc-271e-489a-a26f-a8cb8ec0518b admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:00.822015 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f36ebc-271e-489a-a26f-a8cb8ec0518b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:00.823978 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f36ebc-271e-489a-a26f-a8cb8ec0518b admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:00.912173 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1308/2611] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:20:00 2022] POST /identity/v3/projects => generated 399 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:20:00.916804 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f32b668-87b7-4040-9a48-cb8763e3f5cd admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:00.928260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f32b668-87b7-4040-9a48-cb8763e3f5cd admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:00.998254 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2048eb9-ee54-40f4-85a8-5f5b6b2f5eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:00.999008 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2048eb9-ee54-40f4-85a8-5f5b6b2f5eec admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:00.999370 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2048eb9-ee54-40f4-85a8-5f5b6b2f5eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:00.999714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2048eb9-ee54-40f4-85a8-5f5b6b2f5eec admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:01.000746 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2048eb9-ee54-40f4-85a8-5f5b6b2f5eec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:01.002891 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2048eb9-ee54-40f4-85a8-5f5b6b2f5eec admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:01.111154 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1304/2612] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:20:00 2022] POST /identity/v3/users => generated 300 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:20:01.115419 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f36ebc-271e-489a-a26f-a8cb8ec0518b admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:01.127777 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f36ebc-271e-489a-a26f-a8cb8ec0518b admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:01.132788 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ccbdd-6b7b-4de1-99a5-fd83e460acef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:01.133402 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ccbdd-6b7b-4de1-99a5-fd83e460acef admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:01.133686 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ccbdd-6b7b-4de1-99a5-fd83e460acef admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:01.133964 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ccbdd-6b7b-4de1-99a5-fd83e460acef admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:01.134945 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81ccbdd-6b7b-4de1-99a5-fd83e460acef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:01.136379 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81ccbdd-6b7b-4de1-99a5-fd83e460acef admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:01.147854 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1309/2613] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:20:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:20:01.151532 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2048eb9-ee54-40f4-85a8-5f5b6b2f5eec admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:01.162177 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2048eb9-ee54-40f4-85a8-5f5b6b2f5eec admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:01.167206 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16212b-d137-4526-a02a-55755e7dc877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:01.167824 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16212b-d137-4526-a02a-55755e7dc877 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:01.168966 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16212b-d137-4526-a02a-55755e7dc877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:01.168966 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16212b-d137-4526-a02a-55755e7dc877 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:01.169133 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f16212b-d137-4526-a02a-55755e7dc877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:01.170874 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f16212b-d137-4526-a02a-55755e7dc877 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:01.183859 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1305/2614] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:20:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:20:01.188596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ccbdd-6b7b-4de1-99a5-fd83e460acef admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:01.199549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ccbdd-6b7b-4de1-99a5-fd83e460acef admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:01.205346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa6b70-d62c-4006-87f0-20e393c21cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:01.206274 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa6b70-d62c-4006-87f0-20e393c21cd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:01.206560 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa6b70-d62c-4006-87f0-20e393c21cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:01.206879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa6b70-d62c-4006-87f0-20e393c21cd7 admin admin] PATH_INFO: `/v3/projects/d9b31bae533b4a0ea7440e3421ddc0b0/users/316d8635716b40bc8dd98ef64ae0524d/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:01.221718 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afa6b70-d62c-4006-87f0-20e393c21cd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9b31bae533b4a0ea7440e3421ddc0b0, user_id=316d8635716b40bc8dd98ef64ae0524d, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:01.224418 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afa6b70-d62c-4006-87f0-20e393c21cd7 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:01.355885 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1310/2615] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:20:01 2022] PUT /identity/v3/projects/d9b31bae533b4a0ea7440e3421ddc0b0/users/316d8635716b40bc8dd98ef64ae0524d/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:20:01.358068 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216bfbf-c94b-4804-8f13-eabad1eee620 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:01.358357 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216bfbf-c94b-4804-8f13-eabad1eee620 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:01.358635 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216bfbf-c94b-4804-8f13-eabad1eee620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:01.411419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-d216bfbf-c94b-4804-8f13-eabad1eee620 None None] MFA Rules not processed for user `316d8635716b40bc8dd98ef64ae0524d`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:01.413315 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d216bfbf-c94b-4804-8f13-eabad1eee620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:01.523074 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1306/2616] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:01 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:20:01.541271 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa6b70-d62c-4006-87f0-20e393c21cd7 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:01.549489 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-1afa6b70-d62c-4006-87f0-20e393c21cd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:01.572764 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:01.632180 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa6b70-d62c-4006-87f0-20e393c21cd7 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:01.638747 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4c7fa-c0aa-4228-9eaa-fb0072172545 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:01.640401 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4c7fa-c0aa-4228-9eaa-fb0072172545 service cinder] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:01.640910 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4c7fa-c0aa-4228-9eaa-fb0072172545 service cinder] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:01.641417 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4c7fa-c0aa-4228-9eaa-fb0072172545 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:01.651846 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f4c7fa-c0aa-4228-9eaa-fb0072172545 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:01.653709 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f4c7fa-c0aa-4228-9eaa-fb0072172545 service cinder] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:01.665133 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1311/2617] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:20:01 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:20:01.996757 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d216bfbf-c94b-4804-8f13-eabad1eee620 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:01.998268 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d216bfbf-c94b-4804-8f13-eabad1eee620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:02.062597 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:02.076572 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d216bfbf-c94b-4804-8f13-eabad1eee620 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:02.082973 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507cb689-722b-49e3-ad7d-a46ff1716e51 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59846d88f4314c2382aa5cb8bae5053b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:02.083598 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507cb689-722b-49e3-ad7d-a46ff1716e51 service placement] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:02.083882 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507cb689-722b-49e3-ad7d-a46ff1716e51 service placement] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:02.084159 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507cb689-722b-49e3-ad7d-a46ff1716e51 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:02.086111 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-507cb689-722b-49e3-ad7d-a46ff1716e51 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:02.098061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507cb689-722b-49e3-ad7d-a46ff1716e51 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:02.099889 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507cb689-722b-49e3-ad7d-a46ff1716e51 service placement] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:02.109820 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1307/2618] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:20:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:20:02.157281 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4c7fa-c0aa-4228-9eaa-fb0072172545 service cinder] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:02.158872 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-c9f4c7fa-c0aa-4228-9eaa-fb0072172545 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:02.227666 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4c7fa-c0aa-4228-9eaa-fb0072172545 service cinder] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:02.232653 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d461064-d00a-4111-b309-8b0d94984282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:02.234161 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d461064-d00a-4111-b309-8b0d94984282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:02.234442 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d461064-d00a-4111-b309-8b0d94984282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:02.234718 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d461064-d00a-4111-b309-8b0d94984282 admin admin] PATH_INFO: `/v3/users/f01737ba3200470c8635f335d0338ac8` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:02.235961 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d461064-d00a-4111-b309-8b0d94984282 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f01737ba3200470c8635f335d0338ac8)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:02.238011 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d461064-d00a-4111-b309-8b0d94984282 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:02.445310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-6d461064-d00a-4111-b309-8b0d94984282 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f01737ba3200470c8635f335d0338ac8'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:02.521513 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-6d461064-d00a-4111-b309-8b0d94984282 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f01737ba3200470c8635f335d0338ac8'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:02.522169 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-6d461064-d00a-4111-b309-8b0d94984282 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f01737ba3200470c8635f335d0338ac8'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:02.535420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-6d461064-d00a-4111-b309-8b0d94984282 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f01737ba3200470c8635f335d0338ac8'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:02.537148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-6d461064-d00a-4111-b309-8b0d94984282 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f01737ba3200470c8635f335d0338ac8'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:02.549031 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1312/2619] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:20:02 2022] DELETE /identity/v3/users/f01737ba3200470c8635f335d0338ac8 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:20:02.815570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507cb689-722b-49e3-ad7d-a46ff1716e51 service placement] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:02.827269 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507cb689-722b-49e3-ad7d-a46ff1716e51 service placement] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:02.887674 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5096a532-a560-436d-8dad-fa4437eaccbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:02.888551 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096a532-a560-436d-8dad-fa4437eaccbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:02.888991 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096a532-a560-436d-8dad-fa4437eaccbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:02.889325 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096a532-a560-436d-8dad-fa4437eaccbc admin admin] PATH_INFO: `/v3/projects/2e1ed25abd8e43009dd8f3f4b7ec1864` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:02.890542 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5096a532-a560-436d-8dad-fa4437eaccbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e1ed25abd8e43009dd8f3f4b7ec1864)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:02.892499 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5096a532-a560-436d-8dad-fa4437eaccbc admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:02.989266 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5096a532-a560-436d-8dad-fa4437eaccbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e1ed25abd8e43009dd8f3f4b7ec1864', 'request_id': 'req-5096a532-a560-436d-8dad-fa4437eaccbc'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:03.053227 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5096a532-a560-436d-8dad-fa4437eaccbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e1ed25abd8e43009dd8f3f4b7ec1864', 'request_id': 'req-5096a532-a560-436d-8dad-fa4437eaccbc'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:03.059337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5096a532-a560-436d-8dad-fa4437eaccbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e1ed25abd8e43009dd8f3f4b7ec1864', 'request_id': 'req-5096a532-a560-436d-8dad-fa4437eaccbc'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:03.070799 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5096a532-a560-436d-8dad-fa4437eaccbc admin admin] The token cache is being invalidate because project 2e1ed25abd8e43009dd8f3f4b7ec1864 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:20:03.071110 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5096a532-a560-436d-8dad-fa4437eaccbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:03.071772 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5096a532-a560-436d-8dad-fa4437eaccbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:03.073299 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1308/2620] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:20:02 2022] DELETE /identity/v3/projects/2e1ed25abd8e43009dd8f3f4b7ec1864 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:20:03.078735 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da713d5-c0ed-4b0d-9b97-4fcc44f3cfd5 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:03.079055 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da713d5-c0ed-4b0d-9b97-4fcc44f3cfd5 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:03.079330 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da713d5-c0ed-4b0d-9b97-4fcc44f3cfd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:03.101061 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-2da713d5-c0ed-4b0d-9b97-4fcc44f3cfd5 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:03.102277 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-2da713d5-c0ed-4b0d-9b97-4fcc44f3cfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:03.134701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:03.162323 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1313/2621] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:20:03.170317 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d883a4-1799-4ea8-bb0d-695f56d427d5 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:03.170607 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d883a4-1799-4ea8-bb0d-695f56d427d5 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:03.171596 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d883a4-1799-4ea8-bb0d-695f56d427d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:03.190431 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-10d883a4-1799-4ea8-bb0d-695f56d427d5 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:03.193265 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-10d883a4-1799-4ea8-bb0d-695f56d427d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:03.199596 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1309/2622] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:20:03.204423 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da713d5-c0ed-4b0d-9b97-4fcc44f3cfd5 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:03.214684 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da713d5-c0ed-4b0d-9b97-4fcc44f3cfd5 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:03.220486 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c9dc5-40c7-4147-b167-ece3853163ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:03.221103 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c9dc5-40c7-4147-b167-ece3853163ce admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:03.221389 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c9dc5-40c7-4147-b167-ece3853163ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:03.221668 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c9dc5-40c7-4147-b167-ece3853163ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:03.222398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c9dc5-40c7-4147-b167-ece3853163ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:20:03.222707 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c9dc5-40c7-4147-b167-ece3853163ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:03.224352 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c9dc5-40c7-4147-b167-ece3853163ce admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:03.236483 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1314/2623] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:20:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:20:03.240434 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d883a4-1799-4ea8-bb0d-695f56d427d5 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:03.249842 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d883a4-1799-4ea8-bb0d-695f56d427d5 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:03.254761 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01580a75-1caf-4897-ab5b-3ba0bcc14f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:03.255409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01580a75-1caf-4897-ab5b-3ba0bcc14f88 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:03.255692 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01580a75-1caf-4897-ab5b-3ba0bcc14f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:03.255969 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01580a75-1caf-4897-ab5b-3ba0bcc14f88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:03.256733 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01580a75-1caf-4897-ab5b-3ba0bcc14f88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:03.258684 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01580a75-1caf-4897-ab5b-3ba0bcc14f88 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:03.375455 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1310/2624] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:20:03 2022] POST /identity/v3/projects => generated 371 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:20:03.380039 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c9dc5-40c7-4147-b167-ece3853163ce admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:03.391715 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c9dc5-40c7-4147-b167-ece3853163ce admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:03.448104 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f87d379-8045-49a9-a6ac-994939e0cd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:03.448733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87d379-8045-49a9-a6ac-994939e0cd4d admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:03.449017 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87d379-8045-49a9-a6ac-994939e0cd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:03.449296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87d379-8045-49a9-a6ac-994939e0cd4d admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:03.449959 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f87d379-8045-49a9-a6ac-994939e0cd4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:03.451943 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f87d379-8045-49a9-a6ac-994939e0cd4d admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:03.552923 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1315/2625] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:20:03 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:20:03.555451 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01580a75-1caf-4897-ab5b-3ba0bcc14f88 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:03.566764 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01580a75-1caf-4897-ab5b-3ba0bcc14f88 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:03.572380 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c3471-cba1-4d17-a373-971be6d77f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:03.572995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c3471-cba1-4d17-a373-971be6d77f36 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:03.573278 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c3471-cba1-4d17-a373-971be6d77f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:03.573972 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c3471-cba1-4d17-a373-971be6d77f36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:03.574363 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c3471-cba1-4d17-a373-971be6d77f36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:03.575995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c3471-cba1-4d17-a373-971be6d77f36 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:03.587474 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1311/2626] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:20:03.591602 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f87d379-8045-49a9-a6ac-994939e0cd4d admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:03.601847 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f87d379-8045-49a9-a6ac-994939e0cd4d admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:03.607190 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e4a64-5001-4c99-a386-5f37708c85df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:03.607806 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e4a64-5001-4c99-a386-5f37708c85df admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:03.608089 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e4a64-5001-4c99-a386-5f37708c85df admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:03.608369 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e4a64-5001-4c99-a386-5f37708c85df admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:03.609073 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416e4a64-5001-4c99-a386-5f37708c85df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:03.610689 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416e4a64-5001-4c99-a386-5f37708c85df admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:03.622062 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1316/2627] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:20:03.626716 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c3471-cba1-4d17-a373-971be6d77f36 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:03.636896 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c3471-cba1-4d17-a373-971be6d77f36 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:03.643057 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4ef1c-6e5f-42b4-9bee-a6982e649e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:03.643814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4ef1c-6e5f-42b4-9bee-a6982e649e2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:03.645237 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4ef1c-6e5f-42b4-9bee-a6982e649e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:03.645237 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4ef1c-6e5f-42b4-9bee-a6982e649e2b admin admin] PATH_INFO: `/v3/projects/ac58d12fea0842589f9fb0b50d01af47/users/3c0509f1249b45c3914f59db2dc936a3/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:03.659643 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b4ef1c-6e5f-42b4-9bee-a6982e649e2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac58d12fea0842589f9fb0b50d01af47, user_id=3c0509f1249b45c3914f59db2dc936a3, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:03.662324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b4ef1c-6e5f-42b4-9bee-a6982e649e2b admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:03.744753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1312/2628] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:20:03 2022] PUT /identity/v3/projects/ac58d12fea0842589f9fb0b50d01af47/users/3c0509f1249b45c3914f59db2dc936a3/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:20:03.757074 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1525e6b-318d-4f1e-a3ce-8f0e34269e56 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:03.757365 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1525e6b-318d-4f1e-a3ce-8f0e34269e56 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:03.757647 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1525e6b-318d-4f1e-a3ce-8f0e34269e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:03.784894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-d1525e6b-318d-4f1e-a3ce-8f0e34269e56 None None] MFA Rules not processed for user `3c0509f1249b45c3914f59db2dc936a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:03.786127 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-d1525e6b-318d-4f1e-a3ce-8f0e34269e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:03.851140 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1317/2629] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:20:03.863343 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4ef1c-6e5f-42b4-9bee-a6982e649e2b admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:03.865134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d2b4ef1c-6e5f-42b4-9bee-a6982e649e2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:03.882480 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:03.943363 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4ef1c-6e5f-42b4-9bee-a6982e649e2b admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:03.950736 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf21e9-98ce-4bad-afb8-5187127790cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:03.951560 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf21e9-98ce-4bad-afb8-5187127790cd service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:03.951855 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf21e9-98ce-4bad-afb8-5187127790cd service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:03.952667 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf21e9-98ce-4bad-afb8-5187127790cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:03.965944 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf21e9-98ce-4bad-afb8-5187127790cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:03.967653 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf21e9-98ce-4bad-afb8-5187127790cd service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:03.977694 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1313/2630] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:20:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:20:09.259017 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1525e6b-318d-4f1e-a3ce-8f0e34269e56 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:09.291464 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:09.319670 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1525e6b-318d-4f1e-a3ce-8f0e34269e56 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:09.324833 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a5476c-e6a2-4f24-98d8-46810bf96d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:09.325473 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5476c-e6a2-4f24-98d8-46810bf96d02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:09.325756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5476c-e6a2-4f24-98d8-46810bf96d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:09.326033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5476c-e6a2-4f24-98d8-46810bf96d02 admin admin] PATH_INFO: `/v3/users/3c0509f1249b45c3914f59db2dc936a3` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:09.327249 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a5476c-e6a2-4f24-98d8-46810bf96d02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c0509f1249b45c3914f59db2dc936a3)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:09.335102 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a5476c-e6a2-4f24-98d8-46810bf96d02 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:09.501973 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-b5a5476c-e6a2-4f24-98d8-46810bf96d02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c0509f1249b45c3914f59db2dc936a3'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:09.594186 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-b5a5476c-e6a2-4f24-98d8-46810bf96d02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c0509f1249b45c3914f59db2dc936a3'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:09.594865 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-b5a5476c-e6a2-4f24-98d8-46810bf96d02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c0509f1249b45c3914f59db2dc936a3'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:09.608060 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-b5a5476c-e6a2-4f24-98d8-46810bf96d02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c0509f1249b45c3914f59db2dc936a3'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:09.608699 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-b5a5476c-e6a2-4f24-98d8-46810bf96d02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c0509f1249b45c3914f59db2dc936a3'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:09.621604 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1318/2631] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:20:09 2022] DELETE /identity/v3/users/3c0509f1249b45c3914f59db2dc936a3 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:20:09.626214 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac58cb-8454-41e4-96b9-ae456e51d027 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:09.626670 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac58cb-8454-41e4-96b9-ae456e51d027 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:09.626766 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac58cb-8454-41e4-96b9-ae456e51d027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:09.646548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-eaac58cb-8454-41e4-96b9-ae456e51d027 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:09.647486 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-eaac58cb-8454-41e4-96b9-ae456e51d027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:09.707285 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1314/2632] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:20:09.718920 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a5476c-e6a2-4f24-98d8-46810bf96d02 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:09.720394 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-b5a5476c-e6a2-4f24-98d8-46810bf96d02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:09.778448 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a5476c-e6a2-4f24-98d8-46810bf96d02 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:09.782070 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7140c90-56ba-4b88-991a-32087dda01b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:09.782688 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7140c90-56ba-4b88-991a-32087dda01b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:09.783007 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7140c90-56ba-4b88-991a-32087dda01b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:09.786581 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7140c90-56ba-4b88-991a-32087dda01b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:09.794106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7140c90-56ba-4b88-991a-32087dda01b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:09.795589 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7140c90-56ba-4b88-991a-32087dda01b8 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:09.805375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1319/2633] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:20:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:20:10.209272 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac58cb-8454-41e4-96b9-ae456e51d027 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:10.210831 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-eaac58cb-8454-41e4-96b9-ae456e51d027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:10.223517 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac58cb-8454-41e4-96b9-ae456e51d027 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:10.228278 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7bcd1-5805-4e65-a184-159a237158f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:10.228937 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7bcd1-5805-4e65-a184-159a237158f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:10.229222 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7bcd1-5805-4e65-a184-159a237158f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:10.229502 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7bcd1-5805-4e65-a184-159a237158f9 admin admin] PATH_INFO: `/v3/projects/ac58d12fea0842589f9fb0b50d01af47` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:10.230665 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7bcd1-5805-4e65-a184-159a237158f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac58d12fea0842589f9fb0b50d01af47)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:10.232623 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7bcd1-5805-4e65-a184-159a237158f9 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:10.355404 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-54e7bcd1-5805-4e65-a184-159a237158f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac58d12fea0842589f9fb0b50d01af47', 'request_id': 'req-54e7bcd1-5805-4e65-a184-159a237158f9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:10.418184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-54e7bcd1-5805-4e65-a184-159a237158f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac58d12fea0842589f9fb0b50d01af47', 'request_id': 'req-54e7bcd1-5805-4e65-a184-159a237158f9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:10.424625 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-54e7bcd1-5805-4e65-a184-159a237158f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac58d12fea0842589f9fb0b50d01af47', 'request_id': 'req-54e7bcd1-5805-4e65-a184-159a237158f9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:10.435707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-54e7bcd1-5805-4e65-a184-159a237158f9 admin admin] The token cache is being invalidate because project ac58d12fea0842589f9fb0b50d01af47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:20:10.435996 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-54e7bcd1-5805-4e65-a184-159a237158f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:10.436628 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-54e7bcd1-5805-4e65-a184-159a237158f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:10.438130 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1315/2634] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:20:10 2022] DELETE /identity/v3/projects/ac58d12fea0842589f9fb0b50d01af47 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:20:10.443805 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5197b-156d-4395-b4bd-c05c5df57696 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:10.444297 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5197b-156d-4395-b4bd-c05c5df57696 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:10.444396 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5197b-156d-4395-b4bd-c05c5df57696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:10.465562 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-9ea5197b-156d-4395-b4bd-c05c5df57696 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:10.466788 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-9ea5197b-156d-4395-b4bd-c05c5df57696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:10.598260 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7bcd1-5805-4e65-a184-159a237158f9 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:10.599843 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-54e7bcd1-5805-4e65-a184-159a237158f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:10.628764 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:10.644974 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:10.656975 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1320/2635] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:20:10.664693 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e755c3-8d75-4fc6-883f-97ee3f50cf3b None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:10.666080 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e755c3-8d75-4fc6-883f-97ee3f50cf3b None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:10.666080 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e755c3-8d75-4fc6-883f-97ee3f50cf3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:10.671535 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7bcd1-5805-4e65-a184-159a237158f9 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:10.676684 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015db2bc-82e0-4782-aefc-95c654f8d893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:10.677739 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015db2bc-82e0-4782-aefc-95c654f8d893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:10.677868 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015db2bc-82e0-4782-aefc-95c654f8d893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:10.678127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015db2bc-82e0-4782-aefc-95c654f8d893 admin admin] PATH_INFO: `/v3/users/316d8635716b40bc8dd98ef64ae0524d` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:10.679356 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015db2bc-82e0-4782-aefc-95c654f8d893 admin admin] RBAC: Authorizing `identity:delete_user(user_id=316d8635716b40bc8dd98ef64ae0524d)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:10.681275 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015db2bc-82e0-4782-aefc-95c654f8d893 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:10.685348 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-26e755c3-8d75-4fc6-883f-97ee3f50cf3b None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:10.687367 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-26e755c3-8d75-4fc6-883f-97ee3f50cf3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:10.696188 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1321/2636] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:20:10.700339 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e755c3-8d75-4fc6-883f-97ee3f50cf3b None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:10.709562 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e755c3-8d75-4fc6-883f-97ee3f50cf3b None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:10.714275 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908da2a9-1412-40ab-9802-50ffdba86475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:10.714901 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908da2a9-1412-40ab-9802-50ffdba86475 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:10.715187 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908da2a9-1412-40ab-9802-50ffdba86475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:10.717623 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908da2a9-1412-40ab-9802-50ffdba86475 admin admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:10.717623 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908da2a9-1412-40ab-9802-50ffdba86475 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:20:10.717623 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908da2a9-1412-40ab-9802-50ffdba86475 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:10.718142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908da2a9-1412-40ab-9802-50ffdba86475 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:10.729768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1322/2637] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 05:20:10.733962 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908da2a9-1412-40ab-9802-50ffdba86475 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:10.743772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908da2a9-1412-40ab-9802-50ffdba86475 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:10.748450 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1123e-e928-4867-9ade-923113ed595e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:10.749046 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1123e-e928-4867-9ade-923113ed595e admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:10.749328 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1123e-e928-4867-9ade-923113ed595e admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:10.749607 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1123e-e928-4867-9ade-923113ed595e admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:10.755705 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1123e-e928-4867-9ade-923113ed595e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:10.755705 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1123e-e928-4867-9ade-923113ed595e admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:10.852491 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1323/2638] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:20:10 2022] POST /identity/v3/projects => generated 405 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:20:10.856632 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1123e-e928-4867-9ade-923113ed595e admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:10.888000 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-015db2bc-82e0-4782-aefc-95c654f8d893 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '316d8635716b40bc8dd98ef64ae0524d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:10.916798 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1123e-e928-4867-9ade-923113ed595e admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:10.921519 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742e75d-beac-4e74-a422-715596fe8646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:10.922185 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742e75d-beac-4e74-a422-715596fe8646 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:10.922446 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742e75d-beac-4e74-a422-715596fe8646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:10.922712 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742e75d-beac-4e74-a422-715596fe8646 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:10.923411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d742e75d-beac-4e74-a422-715596fe8646 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:10.929044 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d742e75d-beac-4e74-a422-715596fe8646 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:10.955845 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-015db2bc-82e0-4782-aefc-95c654f8d893 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '316d8635716b40bc8dd98ef64ae0524d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:10.956459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-015db2bc-82e0-4782-aefc-95c654f8d893 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '316d8635716b40bc8dd98ef64ae0524d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:10.969580 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-015db2bc-82e0-4782-aefc-95c654f8d893 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '316d8635716b40bc8dd98ef64ae0524d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:10.970229 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-015db2bc-82e0-4782-aefc-95c654f8d893 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '316d8635716b40bc8dd98ef64ae0524d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:10.982958 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1316/2639] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:20:10 2022] DELETE /identity/v3/users/316d8635716b40bc8dd98ef64ae0524d => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:20:10.987951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2155775-635e-4c26-a222-39400835a19f None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:10.988270 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2155775-635e-4c26-a222-39400835a19f None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:10.988681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2155775-635e-4c26-a222-39400835a19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:11.006968 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-c2155775-635e-4c26-a222-39400835a19f None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:11.008161 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c2155775-635e-4c26-a222-39400835a19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:11.013213 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1324/2640] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:20:10 2022] POST /identity/v3/users => generated 303 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:20:11.017226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742e75d-beac-4e74-a422-715596fe8646 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:11.018959 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-d742e75d-beac-4e74-a422-715596fe8646 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:11.030911 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742e75d-beac-4e74-a422-715596fe8646 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:11.070796 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1317/2641] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:20:11.080966 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2155775-635e-4c26-a222-39400835a19f None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:11.082311 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c2155775-635e-4c26-a222-39400835a19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:11.089083 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c3327-371c-4078-b561-ba039c79f0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:11.089711 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c3327-371c-4078-b561-ba039c79f0f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:11.090005 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c3327-371c-4078-b561-ba039c79f0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:11.090284 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c3327-371c-4078-b561-ba039c79f0f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:11.091012 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c3327-371c-4078-b561-ba039c79f0f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:11.092631 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c3327-371c-4078-b561-ba039c79f0f8 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:11.103088 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1325/2642] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:20:11.107229 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c3327-371c-4078-b561-ba039c79f0f8 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:11.117485 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c3327-371c-4078-b561-ba039c79f0f8 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:11.122481 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e278ccd-d050-4703-bb9c-0889248e46b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:11.123167 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e278ccd-d050-4703-bb9c-0889248e46b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:11.123450 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e278ccd-d050-4703-bb9c-0889248e46b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:11.123726 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e278ccd-d050-4703-bb9c-0889248e46b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:11.124412 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e278ccd-d050-4703-bb9c-0889248e46b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:11.126001 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e278ccd-d050-4703-bb9c-0889248e46b3 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:11.128584 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:11.136965 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1326/2643] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:20:11.141006 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e278ccd-d050-4703-bb9c-0889248e46b3 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:11.141133 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2155775-635e-4c26-a222-39400835a19f None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:11.145714 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e5a1f0-168e-4708-a78e-273a6e25e696 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:11.146333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5a1f0-168e-4708-a78e-273a6e25e696 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:11.146622 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5a1f0-168e-4708-a78e-273a6e25e696 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:11.146917 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5a1f0-168e-4708-a78e-273a6e25e696 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:11.151284 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e278ccd-d050-4703-bb9c-0889248e46b3 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:11.155942 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64916e68-45e3-499d-8926-eda36c4f9b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:11.156565 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e5a1f0-168e-4708-a78e-273a6e25e696 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:11.156683 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64916e68-45e3-499d-8926-eda36c4f9b33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:11.156847 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64916e68-45e3-499d-8926-eda36c4f9b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:11.157120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64916e68-45e3-499d-8926-eda36c4f9b33 admin admin] PATH_INFO: `/v3/projects/7fc30c3c815e4bd5b788de9c7bec1f28/users/30cc4d3ebfae4edfb339734c31509949/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:11.158590 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e5a1f0-168e-4708-a78e-273a6e25e696 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:11.169071 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1318/2644] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:20:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:20:11.172721 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64916e68-45e3-499d-8926-eda36c4f9b33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fc30c3c815e4bd5b788de9c7bec1f28, user_id=30cc4d3ebfae4edfb339734c31509949, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:11.175594 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64916e68-45e3-499d-8926-eda36c4f9b33 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:11.298914 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1327/2645] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:20:11 2022] PUT /identity/v3/projects/7fc30c3c815e4bd5b788de9c7bec1f28/users/30cc4d3ebfae4edfb339734c31509949/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:20:11.310778 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b434e-82a9-4d3b-9012-db0de9fb302a None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:11.311132 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b434e-82a9-4d3b-9012-db0de9fb302a None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:11.311418 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b434e-82a9-4d3b-9012-db0de9fb302a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:11.340027 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-394b434e-82a9-4d3b-9012-db0de9fb302a None None] MFA Rules not processed for user `30cc4d3ebfae4edfb339734c31509949`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:11.341240 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-394b434e-82a9-4d3b-9012-db0de9fb302a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:11.422016 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1319/2646] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:11 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:20:11.426521 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64916e68-45e3-499d-8926-eda36c4f9b33 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:11.432271 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:11.491570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64916e68-45e3-499d-8926-eda36c4f9b33 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:11.496398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b2cc0-4b11-45f8-aa48-98e9295d76f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:11.497047 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b2cc0-4b11-45f8-aa48-98e9295d76f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:11.497457 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b2cc0-4b11-45f8-aa48-98e9295d76f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:11.497740 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b2cc0-4b11-45f8-aa48-98e9295d76f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:11.498457 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836b2cc0-4b11-45f8-aa48-98e9295d76f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:11.500406 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836b2cc0-4b11-45f8-aa48-98e9295d76f5 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:11.597579 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b434e-82a9-4d3b-9012-db0de9fb302a None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:11.599461 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-394b434e-82a9-4d3b-9012-db0de9fb302a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:11.614392 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b434e-82a9-4d3b-9012-db0de9fb302a None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:11.620162 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c922bb-9dcb-4717-bf8b-90490e038ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:11.621033 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c922bb-9dcb-4717-bf8b-90490e038ce2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:11.621317 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c922bb-9dcb-4717-bf8b-90490e038ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:11.621710 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c922bb-9dcb-4717-bf8b-90490e038ce2 admin admin] PATH_INFO: `/v3/projects/d9b31bae533b4a0ea7440e3421ddc0b0` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:11.622822 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c922bb-9dcb-4717-bf8b-90490e038ce2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9b31bae533b4a0ea7440e3421ddc0b0)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:11.624853 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c922bb-9dcb-4717-bf8b-90490e038ce2 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:11.629165 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1328/2647] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:20:11 2022] POST /identity/v3/projects => generated 405 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:20:11.630073 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b2cc0-4b11-45f8-aa48-98e9295d76f5 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:11.694981 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b2cc0-4b11-45f8-aa48-98e9295d76f5 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:11.699758 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c68b5-d087-4ebe-a660-2cb98266b5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:11.700371 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c68b5-d087-4ebe-a660-2cb98266b5ad admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:11.700683 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c68b5-d087-4ebe-a660-2cb98266b5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:11.701059 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c68b5-d087-4ebe-a660-2cb98266b5ad admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:11.701713 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c68b5-d087-4ebe-a660-2cb98266b5ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:11.703674 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c68b5-d087-4ebe-a660-2cb98266b5ad admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:11.731943 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-71c922bb-9dcb-4717-bf8b-90490e038ce2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9b31bae533b4a0ea7440e3421ddc0b0', 'request_id': 'req-71c922bb-9dcb-4717-bf8b-90490e038ce2'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:11.794206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1329/2648] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:20:11 2022] POST /identity/v3/users => generated 309 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:20:11.798082 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c68b5-d087-4ebe-a660-2cb98266b5ad admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:11.808768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c68b5-d087-4ebe-a660-2cb98266b5ad admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:11.816864 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-71c922bb-9dcb-4717-bf8b-90490e038ce2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9b31bae533b4a0ea7440e3421ddc0b0', 'request_id': 'req-71c922bb-9dcb-4717-bf8b-90490e038ce2'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:11.822994 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-71c922bb-9dcb-4717-bf8b-90490e038ce2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9b31bae533b4a0ea7440e3421ddc0b0', 'request_id': 'req-71c922bb-9dcb-4717-bf8b-90490e038ce2'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:11.835828 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-71c922bb-9dcb-4717-bf8b-90490e038ce2 admin admin] The token cache is being invalidate because project d9b31bae533b4a0ea7440e3421ddc0b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:20:11.836126 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-71c922bb-9dcb-4717-bf8b-90490e038ce2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:11.836766 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-71c922bb-9dcb-4717-bf8b-90490e038ce2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:11.838286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1320/2649] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:20:11 2022] DELETE /identity/v3/projects/d9b31bae533b4a0ea7440e3421ddc0b0 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:20:11.843055 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06200c-7265-4a7d-883d-3409e9428bf7 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:11.843343 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06200c-7265-4a7d-883d-3409e9428bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:11.843621 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06200c-7265-4a7d-883d-3409e9428bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:11.864371 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-ec06200c-7265-4a7d-883d-3409e9428bf7 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:11.865638 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ec06200c-7265-4a7d-883d-3409e9428bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:11.871782 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4399f040-27cc-4984-ad0a-ad71973bb9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:11.872411 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399f040-27cc-4984-ad0a-ad71973bb9da admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:11.872694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399f040-27cc-4984-ad0a-ad71973bb9da admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:11.872970 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399f040-27cc-4984-ad0a-ad71973bb9da admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:11.873669 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4399f040-27cc-4984-ad0a-ad71973bb9da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:11.875318 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4399f040-27cc-4984-ad0a-ad71973bb9da admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:11.885962 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1330/2650] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:20:11.889864 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4399f040-27cc-4984-ad0a-ad71973bb9da admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:11.891589 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-4399f040-27cc-4984-ad0a-ad71973bb9da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:11.904034 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4399f040-27cc-4984-ad0a-ad71973bb9da admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:11.908756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a7859e-d606-43dd-8e57-a778cbc2f438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:11.909419 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7859e-d606-43dd-8e57-a778cbc2f438 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:11.909703 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7859e-d606-43dd-8e57-a778cbc2f438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:11.909982 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7859e-d606-43dd-8e57-a778cbc2f438 admin admin] PATH_INFO: `/v3/projects/505a20d163ad47c99f32b0cb33647834/users/68217e42b1db44829ff667a198bc7e4d/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:11.918142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:11.923307 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:11.927225 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a7859e-d606-43dd-8e57-a778cbc2f438 admin admin] RBAC: Authorizing `identity:create_grant(project_id=505a20d163ad47c99f32b0cb33647834, user_id=68217e42b1db44829ff667a198bc7e4d, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:11.931209 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1321/2651] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:20:11.931331 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a7859e-d606-43dd-8e57-a778cbc2f438 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:11.938997 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4e964-37ba-4a75-9ada-1c285badd2ed None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:11.939885 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4e964-37ba-4a75-9ada-1c285badd2ed None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:11.940196 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4e964-37ba-4a75-9ada-1c285badd2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:11.959998 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-82e4e964-37ba-4a75-9ada-1c285badd2ed None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:11.961191 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-82e4e964-37ba-4a75-9ada-1c285badd2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:11.968980 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1322/2652] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:20:11.973106 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4e964-37ba-4a75-9ada-1c285badd2ed None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:11.982031 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4e964-37ba-4a75-9ada-1c285badd2ed None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:11.987128 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278f341-c2df-4bab-8d59-8f12321f3203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:11.987707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278f341-c2df-4bab-8d59-8f12321f3203 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:11.987985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278f341-c2df-4bab-8d59-8f12321f3203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:11.988258 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278f341-c2df-4bab-8d59-8f12321f3203 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:11.988942 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a278f341-c2df-4bab-8d59-8f12321f3203 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:20:11.989238 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a278f341-c2df-4bab-8d59-8f12321f3203 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:11.990814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a278f341-c2df-4bab-8d59-8f12321f3203 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:12.002304 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1323/2653] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 05:20:12.007019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278f341-c2df-4bab-8d59-8f12321f3203 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:12.016649 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278f341-c2df-4bab-8d59-8f12321f3203 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:12.021222 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c289ac88-d5de-4af4-bcd8-35e471384b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:12.021806 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289ac88-d5de-4af4-bcd8-35e471384b4d admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:12.022093 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289ac88-d5de-4af4-bcd8-35e471384b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:12.022372 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289ac88-d5de-4af4-bcd8-35e471384b4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:12.023226 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c289ac88-d5de-4af4-bcd8-35e471384b4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:12.024950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c289ac88-d5de-4af4-bcd8-35e471384b4d admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:12.028303 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1331/2654] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:20:11 2022] PUT /identity/v3/projects/505a20d163ad47c99f32b0cb33647834/users/68217e42b1db44829ff667a198bc7e4d/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:20:12.032199 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a7859e-d606-43dd-8e57-a778cbc2f438 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:12.100535 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a7859e-d606-43dd-8e57-a778cbc2f438 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:12.105520 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b589b8-375d-4012-96e9-bcde70697183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:12.106141 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b589b8-375d-4012-96e9-bcde70697183 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:12.106423 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b589b8-375d-4012-96e9-bcde70697183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:12.106701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b589b8-375d-4012-96e9-bcde70697183 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:12.107424 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b589b8-375d-4012-96e9-bcde70697183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:12.109105 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b589b8-375d-4012-96e9-bcde70697183 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:12.110525 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1324/2655] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:20:12 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:20:12.115274 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c289ac88-d5de-4af4-bcd8-35e471384b4d admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:12.120729 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1332/2656] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:20:12.124757 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b589b8-375d-4012-96e9-bcde70697183 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:12.136140 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b589b8-375d-4012-96e9-bcde70697183 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:12.184952 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c289ac88-d5de-4af4-bcd8-35e471384b4d admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:12.190253 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbee20f-f039-4edd-9acd-a6c37ab2c36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:12.190898 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbee20f-f039-4edd-9acd-a6c37ab2c36d admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:12.191192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbee20f-f039-4edd-9acd-a6c37ab2c36d admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:12.191464 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbee20f-f039-4edd-9acd-a6c37ab2c36d admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:12.192120 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbee20f-f039-4edd-9acd-a6c37ab2c36d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:12.194547 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbee20f-f039-4edd-9acd-a6c37ab2c36d admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:12.196115 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcfe3e7-8671-43fc-bc53-9c12e839d7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:12.196790 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfe3e7-8671-43fc-bc53-9c12e839d7f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:12.197076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfe3e7-8671-43fc-bc53-9c12e839d7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:12.197409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfe3e7-8671-43fc-bc53-9c12e839d7f1 admin admin] PATH_INFO: `/v3/projects/505a20d163ad47c99f32b0cb33647834/users/68217e42b1db44829ff667a198bc7e4d/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:12.199525 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcfe3e7-8671-43fc-bc53-9c12e839d7f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=505a20d163ad47c99f32b0cb33647834, user_id=68217e42b1db44829ff667a198bc7e4d, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:12.203203 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcfe3e7-8671-43fc-bc53-9c12e839d7f1 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:12.213326 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1333/2657] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:20:12 2022] PUT /identity/v3/projects/505a20d163ad47c99f32b0cb33647834/users/68217e42b1db44829ff667a198bc7e4d/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:20:12.223423 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7f48a-d4ef-4cfc-9187-c607a88502b1 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:12.226220 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7f48a-d4ef-4cfc-9187-c607a88502b1 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:12.226220 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7f48a-d4ef-4cfc-9187-c607a88502b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:12.254705 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-4ef7f48a-d4ef-4cfc-9187-c607a88502b1 None None] MFA Rules not processed for user `68217e42b1db44829ff667a198bc7e4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:12.255943 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-4ef7f48a-d4ef-4cfc-9187-c607a88502b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:12.280525 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1325/2658] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:20:12 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:20:12.285209 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbee20f-f039-4edd-9acd-a6c37ab2c36d admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:12.294741 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbee20f-f039-4edd-9acd-a6c37ab2c36d admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:12.323711 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1334/2659] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:12 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:20:12.334514 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7f48a-d4ef-4cfc-9187-c607a88502b1 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:12.335921 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-4ef7f48a-d4ef-4cfc-9187-c607a88502b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:12.346367 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:12.352836 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d7a06-2a9a-4184-b548-7725467bf5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:12.353471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d7a06-2a9a-4184-b548-7725467bf5f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:12.353754 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d7a06-2a9a-4184-b548-7725467bf5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:12.354030 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d7a06-2a9a-4184-b548-7725467bf5f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:12.354765 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d7a06-2a9a-4184-b548-7725467bf5f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:12.356407 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d7a06-2a9a-4184-b548-7725467bf5f1 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:12.367058 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1326/2660] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:20:12.371118 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d7a06-2a9a-4184-b548-7725467bf5f1 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:12.381357 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d7a06-2a9a-4184-b548-7725467bf5f1 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:12.386137 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4be21-0a9a-4e14-a8ae-e94dabf30050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:12.386773 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4be21-0a9a-4e14-a8ae-e94dabf30050 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:12.387085 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4be21-0a9a-4e14-a8ae-e94dabf30050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:12.387368 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4be21-0a9a-4e14-a8ae-e94dabf30050 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:12.388086 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce4be21-0a9a-4e14-a8ae-e94dabf30050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:12.389727 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce4be21-0a9a-4e14-a8ae-e94dabf30050 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:12.397434 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7f48a-d4ef-4cfc-9187-c607a88502b1 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:12.401995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1327/2661] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:20:12.402435 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762963eb-cbcf-4564-83ce-54437fb89600 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:12.403189 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762963eb-cbcf-4564-83ce-54437fb89600 service cinder] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:12.403496 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762963eb-cbcf-4564-83ce-54437fb89600 service cinder] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:12.403806 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762963eb-cbcf-4564-83ce-54437fb89600 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:12.410325 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4be21-0a9a-4e14-a8ae-e94dabf30050 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:12.418905 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762963eb-cbcf-4564-83ce-54437fb89600 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:12.420170 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4be21-0a9a-4e14-a8ae-e94dabf30050 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:12.420552 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762963eb-cbcf-4564-83ce-54437fb89600 service cinder] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:12.425431 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f66eb-7142-4ba1-9c54-9c66138db623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:12.426092 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f66eb-7142-4ba1-9c54-9c66138db623 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:12.426379 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f66eb-7142-4ba1-9c54-9c66138db623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:12.426658 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f66eb-7142-4ba1-9c54-9c66138db623 admin admin] PATH_INFO: `/v3/projects/3c986ef0f5ea46f9b5e788aaca042b06/users/ac3954ab39014ef3819111a6fe8142b7/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:12.431143 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1335/2662] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:20:12 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:20:12.441848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f66eb-7142-4ba1-9c54-9c66138db623 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c986ef0f5ea46f9b5e788aaca042b06, user_id=ac3954ab39014ef3819111a6fe8142b7, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:12.444580 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f66eb-7142-4ba1-9c54-9c66138db623 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:12.528035 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1328/2663] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:20:12 2022] PUT /identity/v3/projects/3c986ef0f5ea46f9b5e788aaca042b06/users/ac3954ab39014ef3819111a6fe8142b7/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:20:12.533871 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c573249-c72e-4450-b57a-b6b2e22b40de None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:12.534346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c573249-c72e-4450-b57a-b6b2e22b40de None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:12.534346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c573249-c72e-4450-b57a-b6b2e22b40de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:12.554719 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-5c573249-c72e-4450-b57a-b6b2e22b40de None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:12.555978 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-5c573249-c72e-4450-b57a-b6b2e22b40de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:12.568842 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f66eb-7142-4ba1-9c54-9c66138db623 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:12.577913 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:12.620687 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1336/2664] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:20:12.630544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c573249-c72e-4450-b57a-b6b2e22b40de None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:12.631920 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-5c573249-c72e-4450-b57a-b6b2e22b40de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:12.636493 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f66eb-7142-4ba1-9c54-9c66138db623 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:12.641295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63640390-7daf-4ba6-bb3f-00e6037345f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:12.641939 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63640390-7daf-4ba6-bb3f-00e6037345f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:12.642875 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63640390-7daf-4ba6-bb3f-00e6037345f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:12.643108 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63640390-7daf-4ba6-bb3f-00e6037345f0 admin admin] PATH_INFO: `/v3/users/30cc4d3ebfae4edfb339734c31509949` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:12.644329 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63640390-7daf-4ba6-bb3f-00e6037345f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30cc4d3ebfae4edfb339734c31509949)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:12.646276 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63640390-7daf-4ba6-bb3f-00e6037345f0 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:12.693004 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c573249-c72e-4450-b57a-b6b2e22b40de None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:12.696676 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec609c22-09dd-4032-a082-53b19d0c209a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:12.697368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec609c22-09dd-4032-a082-53b19d0c209a service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:12.697655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec609c22-09dd-4032-a082-53b19d0c209a service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:12.697933 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec609c22-09dd-4032-a082-53b19d0c209a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:12.712580 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec609c22-09dd-4032-a082-53b19d0c209a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:12.714042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec609c22-09dd-4032-a082-53b19d0c209a service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:12.724555 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1337/2665] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:20:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:20:12.850185 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-63640390-7daf-4ba6-bb3f-00e6037345f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30cc4d3ebfae4edfb339734c31509949'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:12.945078 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-63640390-7daf-4ba6-bb3f-00e6037345f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30cc4d3ebfae4edfb339734c31509949'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:12.945740 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-63640390-7daf-4ba6-bb3f-00e6037345f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30cc4d3ebfae4edfb339734c31509949'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:12.963351 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-63640390-7daf-4ba6-bb3f-00e6037345f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30cc4d3ebfae4edfb339734c31509949'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:12.964110 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-63640390-7daf-4ba6-bb3f-00e6037345f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30cc4d3ebfae4edfb339734c31509949'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:12.983438 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1329/2666] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:20:12 2022] DELETE /identity/v3/users/30cc4d3ebfae4edfb339734c31509949 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:20:12.988058 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315eb074-b948-4753-ad6a-d9ca6c82a3dd None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:12.988355 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315eb074-b948-4753-ad6a-d9ca6c82a3dd None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:12.988629 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315eb074-b948-4753-ad6a-d9ca6c82a3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:13.007527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-315eb074-b948-4753-ad6a-d9ca6c82a3dd None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:13.008719 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-315eb074-b948-4753-ad6a-d9ca6c82a3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:13.057978 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=100 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:13.072162 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1338/2667] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:20:13.082684 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63640390-7daf-4ba6-bb3f-00e6037345f0 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:13.084386 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-63640390-7daf-4ba6-bb3f-00e6037345f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:13.143829 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63640390-7daf-4ba6-bb3f-00e6037345f0 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:13.147502 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d597bd5-408d-413b-ad4a-83fec7558e99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:13.148119 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d597bd5-408d-413b-ad4a-83fec7558e99 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:13.148435 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d597bd5-408d-413b-ad4a-83fec7558e99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:13.148712 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d597bd5-408d-413b-ad4a-83fec7558e99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:13.158470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d597bd5-408d-413b-ad4a-83fec7558e99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:13.159980 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d597bd5-408d-413b-ad4a-83fec7558e99 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:13.169461 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1330/2668] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:20:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:20:13.561877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315eb074-b948-4753-ad6a-d9ca6c82a3dd None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:13.563444 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-315eb074-b948-4753-ad6a-d9ca6c82a3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:13.592781 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315eb074-b948-4753-ad6a-d9ca6c82a3dd None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:13.599470 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a08856-c85e-44c2-bc58-3e9aea4edf89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:13.600117 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a08856-c85e-44c2-bc58-3e9aea4edf89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:13.600400 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a08856-c85e-44c2-bc58-3e9aea4edf89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:13.600675 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a08856-c85e-44c2-bc58-3e9aea4edf89 admin admin] PATH_INFO: `/v3/projects/7fc30c3c815e4bd5b788de9c7bec1f28` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:13.601875 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a08856-c85e-44c2-bc58-3e9aea4edf89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fc30c3c815e4bd5b788de9c7bec1f28)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:13.603836 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a08856-c85e-44c2-bc58-3e9aea4edf89 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:13.710486 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-d3a08856-c85e-44c2-bc58-3e9aea4edf89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fc30c3c815e4bd5b788de9c7bec1f28', 'request_id': 'req-d3a08856-c85e-44c2-bc58-3e9aea4edf89'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:13.716618 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-d3a08856-c85e-44c2-bc58-3e9aea4edf89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fc30c3c815e4bd5b788de9c7bec1f28', 'request_id': 'req-d3a08856-c85e-44c2-bc58-3e9aea4edf89'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:13.778618 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-d3a08856-c85e-44c2-bc58-3e9aea4edf89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fc30c3c815e4bd5b788de9c7bec1f28', 'request_id': 'req-d3a08856-c85e-44c2-bc58-3e9aea4edf89'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:13.790491 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-d3a08856-c85e-44c2-bc58-3e9aea4edf89 admin admin] The token cache is being invalidate because project 7fc30c3c815e4bd5b788de9c7bec1f28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:20:13.790925 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-d3a08856-c85e-44c2-bc58-3e9aea4edf89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:13.791562 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-d3a08856-c85e-44c2-bc58-3e9aea4edf89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:13.793158 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1339/2669] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:20:13 2022] DELETE /identity/v3/projects/7fc30c3c815e4bd5b788de9c7bec1f28 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:20:13.797297 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d597bd5-408d-413b-ad4a-83fec7558e99 service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:13.812408 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d597bd5-408d-413b-ad4a-83fec7558e99 service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:13.841396 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:13.871735 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc852baf-4e17-4421-9d1f-fea87870f846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:13.872373 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc852baf-4e17-4421-9d1f-fea87870f846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:13.872633 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc852baf-4e17-4421-9d1f-fea87870f846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:13.872906 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc852baf-4e17-4421-9d1f-fea87870f846 admin admin] PATH_INFO: `/v3/users/68217e42b1db44829ff667a198bc7e4d` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:13.874052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc852baf-4e17-4421-9d1f-fea87870f846 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68217e42b1db44829ff667a198bc7e4d)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:13.876023 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc852baf-4e17-4421-9d1f-fea87870f846 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:14.085512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-cc852baf-4e17-4421-9d1f-fea87870f846 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68217e42b1db44829ff667a198bc7e4d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:14.161426 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-cc852baf-4e17-4421-9d1f-fea87870f846 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68217e42b1db44829ff667a198bc7e4d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:14.162554 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-cc852baf-4e17-4421-9d1f-fea87870f846 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68217e42b1db44829ff667a198bc7e4d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:14.176755 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-cc852baf-4e17-4421-9d1f-fea87870f846 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68217e42b1db44829ff667a198bc7e4d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:14.177681 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-cc852baf-4e17-4421-9d1f-fea87870f846 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68217e42b1db44829ff667a198bc7e4d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:14.190708 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1331/2670] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:20:13 2022] DELETE /identity/v3/users/68217e42b1db44829ff667a198bc7e4d => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:20:14.681728 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a08856-c85e-44c2-bc58-3e9aea4edf89 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:14.683358 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-d3a08856-c85e-44c2-bc58-3e9aea4edf89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:14.711418 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a08856-c85e-44c2-bc58-3e9aea4edf89 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:14.781525 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4efcd-a778-49eb-be3e-ce00a2e74423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:14.782346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4efcd-a778-49eb-be3e-ce00a2e74423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:14.782629 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4efcd-a778-49eb-be3e-ce00a2e74423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:14.782976 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4efcd-a778-49eb-be3e-ce00a2e74423 admin admin] PATH_INFO: `/v3/projects/505a20d163ad47c99f32b0cb33647834` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:14.784145 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb4efcd-a778-49eb-be3e-ce00a2e74423 admin admin] RBAC: Authorizing `identity:delete_project(project_id=505a20d163ad47c99f32b0cb33647834)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:14.786064 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb4efcd-a778-49eb-be3e-ce00a2e74423 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:14.926285 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-9fb4efcd-a778-49eb-be3e-ce00a2e74423 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '505a20d163ad47c99f32b0cb33647834', 'request_id': 'req-9fb4efcd-a778-49eb-be3e-ce00a2e74423'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:14.933621 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-9fb4efcd-a778-49eb-be3e-ce00a2e74423 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '505a20d163ad47c99f32b0cb33647834', 'request_id': 'req-9fb4efcd-a778-49eb-be3e-ce00a2e74423'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:14.989377 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-9fb4efcd-a778-49eb-be3e-ce00a2e74423 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '505a20d163ad47c99f32b0cb33647834', 'request_id': 'req-9fb4efcd-a778-49eb-be3e-ce00a2e74423'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:15.010334 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-9fb4efcd-a778-49eb-be3e-ce00a2e74423 admin admin] The token cache is being invalidate because project 505a20d163ad47c99f32b0cb33647834 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:20:15.010870 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-9fb4efcd-a778-49eb-be3e-ce00a2e74423 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:15.011724 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-9fb4efcd-a778-49eb-be3e-ce00a2e74423 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:15.013507 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1340/2671] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:20:14 2022] DELETE /identity/v3/projects/505a20d163ad47c99f32b0cb33647834 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:20:15.021061 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0e78c-62b4-45cf-90e9-7ed668495ea5 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:15.024400 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0e78c-62b4-45cf-90e9-7ed668495ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:15.024400 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0e78c-62b4-45cf-90e9-7ed668495ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:15.076740 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-9cc0e78c-62b4-45cf-90e9-7ed668495ea5 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:15.078278 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9cc0e78c-62b4-45cf-90e9-7ed668495ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:15.168157 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1332/2672] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:20:15.176712 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9626b398-ba1e-4425-abc8-969ce9b42218 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:15.177006 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9626b398-ba1e-4425-abc8-969ce9b42218 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:15.177290 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9626b398-ba1e-4425-abc8-969ce9b42218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:15.193061 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:15.199236 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-9626b398-ba1e-4425-abc8-969ce9b42218 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:15.200773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-9626b398-ba1e-4425-abc8-969ce9b42218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:15.209273 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1341/2673] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:20:15.213731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0e78c-62b4-45cf-90e9-7ed668495ea5 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:15.224029 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0e78c-62b4-45cf-90e9-7ed668495ea5 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:15.228793 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04744eb0-9751-4942-9298-10bd9be3c811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:15.229409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04744eb0-9751-4942-9298-10bd9be3c811 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:15.229697 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04744eb0-9751-4942-9298-10bd9be3c811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:15.229978 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04744eb0-9751-4942-9298-10bd9be3c811 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:15.230677 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04744eb0-9751-4942-9298-10bd9be3c811 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:20:15.231019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04744eb0-9751-4942-9298-10bd9be3c811 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:15.232642 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04744eb0-9751-4942-9298-10bd9be3c811 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:15.245715 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1333/2674] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:20:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:20:15.250085 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9626b398-ba1e-4425-abc8-969ce9b42218 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:15.259962 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9626b398-ba1e-4425-abc8-969ce9b42218 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:15.265032 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae231f-13a1-448c-be45-34274ab71871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:15.265760 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae231f-13a1-448c-be45-34274ab71871 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:15.266147 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae231f-13a1-448c-be45-34274ab71871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:15.266527 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae231f-13a1-448c-be45-34274ab71871 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:15.267340 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae231f-13a1-448c-be45-34274ab71871 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:15.269452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae231f-13a1-448c-be45-34274ab71871 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:15.346433 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1342/2675] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:20:15 2022] POST /identity/v3/projects => generated 405 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:20:15.351523 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04744eb0-9751-4942-9298-10bd9be3c811 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:15.363143 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04744eb0-9751-4942-9298-10bd9be3c811 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:15.399528 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:15.448794 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539440ed-80fe-4e98-b54f-96d97cf4299a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:15.449632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539440ed-80fe-4e98-b54f-96d97cf4299a admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:15.450059 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539440ed-80fe-4e98-b54f-96d97cf4299a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:15.450340 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539440ed-80fe-4e98-b54f-96d97cf4299a admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:15.451093 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539440ed-80fe-4e98-b54f-96d97cf4299a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:15.453255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539440ed-80fe-4e98-b54f-96d97cf4299a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:15.556676 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1334/2676] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:20:15 2022] POST /identity/v3/users => generated 303 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:20:15.561254 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae231f-13a1-448c-be45-34274ab71871 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:15.573429 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae231f-13a1-448c-be45-34274ab71871 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:15.582023 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413bf1d0-ab6f-45d7-b3bf-c1b17a3ceaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:15.583509 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413bf1d0-ab6f-45d7-b3bf-c1b17a3ceaa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:15.583994 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413bf1d0-ab6f-45d7-b3bf-c1b17a3ceaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:15.584638 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413bf1d0-ab6f-45d7-b3bf-c1b17a3ceaa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:15.585531 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413bf1d0-ab6f-45d7-b3bf-c1b17a3ceaa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:15.587580 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413bf1d0-ab6f-45d7-b3bf-c1b17a3ceaa8 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:15.604988 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1343/2677] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:20:15 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:20:15.608751 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539440ed-80fe-4e98-b54f-96d97cf4299a admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:15.621575 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539440ed-80fe-4e98-b54f-96d97cf4299a admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:15.627373 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf21507-8206-4a06-a37b-5456ca1adc5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:15.628226 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf21507-8206-4a06-a37b-5456ca1adc5d admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:15.628525 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf21507-8206-4a06-a37b-5456ca1adc5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:15.628860 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf21507-8206-4a06-a37b-5456ca1adc5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:15.629644 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf21507-8206-4a06-a37b-5456ca1adc5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:15.631799 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf21507-8206-4a06-a37b-5456ca1adc5d admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:15.645524 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1335/2678] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:20:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:20:15.649880 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413bf1d0-ab6f-45d7-b3bf-c1b17a3ceaa8 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:15.665321 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413bf1d0-ab6f-45d7-b3bf-c1b17a3ceaa8 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:15.673926 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd05bd48-343c-464f-8696-3f194007b635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:15.675013 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd05bd48-343c-464f-8696-3f194007b635 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:15.675263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd05bd48-343c-464f-8696-3f194007b635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:15.675546 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd05bd48-343c-464f-8696-3f194007b635 admin admin] PATH_INFO: `/v3/projects/1280353c68cf4f8388ea94ba12d1ddcd/users/c13e7a596684498f930cf6b61a2b6124/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:15.692036 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd05bd48-343c-464f-8696-3f194007b635 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1280353c68cf4f8388ea94ba12d1ddcd, user_id=c13e7a596684498f930cf6b61a2b6124, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:15.694745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd05bd48-343c-464f-8696-3f194007b635 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:15.768747 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1344/2679] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:20:15 2022] PUT /identity/v3/projects/1280353c68cf4f8388ea94ba12d1ddcd/users/c13e7a596684498f930cf6b61a2b6124/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:20:15.774324 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff623f-da8e-4217-a386-10a839cf201b None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:15.774610 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff623f-da8e-4217-a386-10a839cf201b None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:15.774936 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff623f-da8e-4217-a386-10a839cf201b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:15.795563 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-27ff623f-da8e-4217-a386-10a839cf201b None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:15.797387 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-27ff623f-da8e-4217-a386-10a839cf201b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:15.866938 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1336/2680] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:20:15.878152 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd05bd48-343c-464f-8696-3f194007b635 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:15.880308 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-cd05bd48-343c-464f-8696-3f194007b635 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:15.959733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd05bd48-343c-464f-8696-3f194007b635 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:15.963997 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2ff01-0e3b-4836-8e06-c79f297a0d81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:15.964630 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2ff01-0e3b-4836-8e06-c79f297a0d81 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:15.964922 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2ff01-0e3b-4836-8e06-c79f297a0d81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:15.965200 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2ff01-0e3b-4836-8e06-c79f297a0d81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:15.977104 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2ff01-0e3b-4836-8e06-c79f297a0d81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:15.978610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2ff01-0e3b-4836-8e06-c79f297a0d81 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:15.989785 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1345/2681] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:20:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:20:17.500638 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40c753-32ba-4af7-a28e-c25ca9a25677 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:17.501130 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40c753-32ba-4af7-a28e-c25ca9a25677 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:17.501234 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40c753-32ba-4af7-a28e-c25ca9a25677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:17.533329 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-ce40c753-32ba-4af7-a28e-c25ca9a25677 None None] MFA Rules not processed for user `c13e7a596684498f930cf6b61a2b6124`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:17.534566 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ce40c753-32ba-4af7-a28e-c25ca9a25677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:17.618690 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1337/2682] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:17 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:20:17.623266 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2ff01-0e3b-4836-8e06-c79f297a0d81 service neutron] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:17.639033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2ff01-0e3b-4836-8e06-c79f297a0d81 service neutron] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:17.650440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a57f683-afa3-4991-9d8f-b6949c64aec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:17.651125 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a57f683-afa3-4991-9d8f-b6949c64aec7 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:17.651414 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a57f683-afa3-4991-9d8f-b6949c64aec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:17.651689 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a57f683-afa3-4991-9d8f-b6949c64aec7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:17.652398 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a57f683-afa3-4991-9d8f-b6949c64aec7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:17.654444 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a57f683-afa3-4991-9d8f-b6949c64aec7 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:17.770076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1346/2683] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:20:17 2022] POST /identity/v3/projects => generated 403 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:20:17.774548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40c753-32ba-4af7-a28e-c25ca9a25677 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:17.785805 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40c753-32ba-4af7-a28e-c25ca9a25677 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:17.853563 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:17.875966 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f42383-14c9-4c7a-8450-4b9c5d4d746c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:17.877592 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f42383-14c9-4c7a-8450-4b9c5d4d746c admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:17.877970 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f42383-14c9-4c7a-8450-4b9c5d4d746c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:17.878174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f42383-14c9-4c7a-8450-4b9c5d4d746c admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:17.878928 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f42383-14c9-4c7a-8450-4b9c5d4d746c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:17.884153 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f42383-14c9-4c7a-8450-4b9c5d4d746c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:17.993967 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1338/2684] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:20:17 2022] POST /identity/v3/users => generated 308 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:20:18.000382 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a57f683-afa3-4991-9d8f-b6949c64aec7 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:18.012026 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a57f683-afa3-4991-9d8f-b6949c64aec7 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:18.016651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85156eac-c8f5-469d-a8bd-dc177ba2bf26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:18.017803 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85156eac-c8f5-469d-a8bd-dc177ba2bf26 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:18.018082 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85156eac-c8f5-469d-a8bd-dc177ba2bf26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:18.018358 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85156eac-c8f5-469d-a8bd-dc177ba2bf26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:18.021719 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85156eac-c8f5-469d-a8bd-dc177ba2bf26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:18.024195 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85156eac-c8f5-469d-a8bd-dc177ba2bf26 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:18.037407 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1347/2685] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:20:18.040914 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f42383-14c9-4c7a-8450-4b9c5d4d746c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:18.051846 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f42383-14c9-4c7a-8450-4b9c5d4d746c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:18.056792 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c58674-288d-417a-a0ba-ce2bc8ed4aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:18.057483 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c58674-288d-417a-a0ba-ce2bc8ed4aff admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:18.057766 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c58674-288d-417a-a0ba-ce2bc8ed4aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:18.058041 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c58674-288d-417a-a0ba-ce2bc8ed4aff admin admin] PATH_INFO: `/v3/projects/4fa014778b1f455180a91fe7c3a6be99/users/9084137934954072900c7766abde762e/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:18.074498 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c58674-288d-417a-a0ba-ce2bc8ed4aff admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fa014778b1f455180a91fe7c3a6be99, user_id=9084137934954072900c7766abde762e, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:18.077517 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c58674-288d-417a-a0ba-ce2bc8ed4aff admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:18.185917 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1339/2686] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:20:18 2022] PUT /identity/v3/projects/4fa014778b1f455180a91fe7c3a6be99/users/9084137934954072900c7766abde762e/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:20:18.189962 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85156eac-c8f5-469d-a8bd-dc177ba2bf26 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:18.201912 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85156eac-c8f5-469d-a8bd-dc177ba2bf26 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:18.275764 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c786b8-82be-4fa1-8b72-20c4d37eb284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:18.277357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c786b8-82be-4fa1-8b72-20c4d37eb284 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:18.277638 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c786b8-82be-4fa1-8b72-20c4d37eb284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:18.277912 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c786b8-82be-4fa1-8b72-20c4d37eb284 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:18.278803 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c786b8-82be-4fa1-8b72-20c4d37eb284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:18.281797 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c786b8-82be-4fa1-8b72-20c4d37eb284 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:18.293251 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1348/2687] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:20:18.297548 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c58674-288d-417a-a0ba-ce2bc8ed4aff admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:18.309866 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c58674-288d-417a-a0ba-ce2bc8ed4aff admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:18.316181 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be138f7-03c0-4ea3-9fe7-ab13e6a2983a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:18.316877 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be138f7-03c0-4ea3-9fe7-ab13e6a2983a admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:18.317165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be138f7-03c0-4ea3-9fe7-ab13e6a2983a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:18.317600 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be138f7-03c0-4ea3-9fe7-ab13e6a2983a admin admin] PATH_INFO: `/v3/projects/4fa014778b1f455180a91fe7c3a6be99/users/9084137934954072900c7766abde762e/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:18.320499 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be138f7-03c0-4ea3-9fe7-ab13e6a2983a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fa014778b1f455180a91fe7c3a6be99, user_id=9084137934954072900c7766abde762e, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:18.323286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be138f7-03c0-4ea3-9fe7-ab13e6a2983a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:18.346109 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1340/2688] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:20:18 2022] PUT /identity/v3/projects/4fa014778b1f455180a91fe7c3a6be99/users/9084137934954072900c7766abde762e/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:20:18.551359 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea267d-bdfa-460e-a5cb-7cccc4c19224 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:18.551726 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea267d-bdfa-460e-a5cb-7cccc4c19224 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:18.552023 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea267d-bdfa-460e-a5cb-7cccc4c19224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:18.573895 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:18.584137 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-5cea267d-bdfa-460e-a5cb-7cccc4c19224 None None] MFA Rules not processed for user `ac3954ab39014ef3819111a6fe8142b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:18.585396 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-5cea267d-bdfa-460e-a5cb-7cccc4c19224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:18.672245 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1349/2689] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:20:18.677272 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be138f7-03c0-4ea3-9fe7-ab13e6a2983a admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:18.679069 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-4be138f7-03c0-4ea3-9fe7-ab13e6a2983a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:18.756488 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be138f7-03c0-4ea3-9fe7-ab13e6a2983a admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:18.761391 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d752ce3-a4db-45e5-973f-40f628becd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:18.762011 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d752ce3-a4db-45e5-973f-40f628becd39 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:18.762292 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d752ce3-a4db-45e5-973f-40f628becd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:18.762571 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d752ce3-a4db-45e5-973f-40f628becd39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:18.763289 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d752ce3-a4db-45e5-973f-40f628becd39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:18.765287 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d752ce3-a4db-45e5-973f-40f628becd39 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:18.849100 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1341/2690] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:20:18 2022] POST /identity/v3/projects => generated 379 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:20:18.853416 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea267d-bdfa-460e-a5cb-7cccc4c19224 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:18.865395 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea267d-bdfa-460e-a5cb-7cccc4c19224 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:18.932834 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358e940-e267-459d-aab9-6f39dc889ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:18.933546 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358e940-e267-459d-aab9-6f39dc889ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:18.933829 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358e940-e267-459d-aab9-6f39dc889ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:18.934106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358e940-e267-459d-aab9-6f39dc889ec7 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:18.934775 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358e940-e267-459d-aab9-6f39dc889ec7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:18.936750 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358e940-e267-459d-aab9-6f39dc889ec7 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:19.057290 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1350/2691] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:20:18 2022] POST /identity/v3/users => generated 296 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:20:19.061411 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d752ce3-a4db-45e5-973f-40f628becd39 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:19.073143 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d752ce3-a4db-45e5-973f-40f628becd39 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:19.087150 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe1a5c-4efe-4bdd-b520-0180adac9a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:19.087657 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe1a5c-4efe-4bdd-b520-0180adac9a7f admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:19.087943 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe1a5c-4efe-4bdd-b520-0180adac9a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:19.088221 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe1a5c-4efe-4bdd-b520-0180adac9a7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:19.088961 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe1a5c-4efe-4bdd-b520-0180adac9a7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:19.090617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe1a5c-4efe-4bdd-b520-0180adac9a7f admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:19.103738 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1342/2692] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:20:19.107538 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358e940-e267-459d-aab9-6f39dc889ec7 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:19.121592 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358e940-e267-459d-aab9-6f39dc889ec7 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:19.124673 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce8718e-a637-44d8-aea2-d78b93758ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:19.125438 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8718e-a637-44d8-aea2-d78b93758ac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:19.125795 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8718e-a637-44d8-aea2-d78b93758ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:19.126184 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8718e-a637-44d8-aea2-d78b93758ac7 admin admin] PATH_INFO: `/v3/projects/f3d47ea13c2d4e9181faf641275f034f/users/f2d6f433ebaa40d2bc5dfaba438b676f/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:19.146615 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce8718e-a637-44d8-aea2-d78b93758ac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3d47ea13c2d4e9181faf641275f034f, user_id=f2d6f433ebaa40d2bc5dfaba438b676f, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:19.149352 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce8718e-a637-44d8-aea2-d78b93758ac7 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:19.271136 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1351/2693] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:20:19 2022] PUT /identity/v3/projects/f3d47ea13c2d4e9181faf641275f034f/users/f2d6f433ebaa40d2bc5dfaba438b676f/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:20:19.275443 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe1a5c-4efe-4bdd-b520-0180adac9a7f admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:19.285772 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe1a5c-4efe-4bdd-b520-0180adac9a7f admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:19.347153 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d25738-394c-4aba-b75b-6840fb946b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:19.347847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d25738-394c-4aba-b75b-6840fb946b40 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:19.348125 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d25738-394c-4aba-b75b-6840fb946b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:19.348528 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d25738-394c-4aba-b75b-6840fb946b40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:19.349235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d25738-394c-4aba-b75b-6840fb946b40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:19.351088 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d25738-394c-4aba-b75b-6840fb946b40 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:19.362127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1343/2694] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:20:19.366404 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce8718e-a637-44d8-aea2-d78b93758ac7 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:19.382732 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce8718e-a637-44d8-aea2-d78b93758ac7 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:19.388344 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17407db-0fb5-4809-b83d-30a6235fc701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:19.389072 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17407db-0fb5-4809-b83d-30a6235fc701 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:19.389318 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17407db-0fb5-4809-b83d-30a6235fc701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:19.389669 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17407db-0fb5-4809-b83d-30a6235fc701 admin admin] PATH_INFO: `/v3/projects/f3d47ea13c2d4e9181faf641275f034f/users/f2d6f433ebaa40d2bc5dfaba438b676f/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:19.393850 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17407db-0fb5-4809-b83d-30a6235fc701 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3d47ea13c2d4e9181faf641275f034f, user_id=f2d6f433ebaa40d2bc5dfaba438b676f, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:19.397301 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17407db-0fb5-4809-b83d-30a6235fc701 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:19.427296 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1352/2695] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:20:19 2022] PUT /identity/v3/projects/f3d47ea13c2d4e9181faf641275f034f/users/f2d6f433ebaa40d2bc5dfaba438b676f/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:20:19.687639 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf161d-caa9-4f09-8d2d-0402cef48a8d None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:19.688173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf161d-caa9-4f09-8d2d-0402cef48a8d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:19.688173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf161d-caa9-4f09-8d2d-0402cef48a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:19.703545 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:19.722959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-6ebf161d-caa9-4f09-8d2d-0402cef48a8d None None] MFA Rules not processed for user `9084137934954072900c7766abde762e`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:19.725480 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6ebf161d-caa9-4f09-8d2d-0402cef48a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:19.816953 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1344/2696] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:19 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:20:21.932181 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17407db-0fb5-4809-b83d-30a6235fc701 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:22.004773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:22.020386 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17407db-0fb5-4809-b83d-30a6235fc701 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:22.027379 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8118c-c82b-4eae-b85e-deb845c86c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:22.028096 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8118c-c82b-4eae-b85e-deb845c86c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:22.028379 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8118c-c82b-4eae-b85e-deb845c86c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:22.028873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8118c-c82b-4eae-b85e-deb845c86c8b admin admin] PATH_INFO: `/v3/users/c13e7a596684498f930cf6b61a2b6124` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:22.029868 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8118c-c82b-4eae-b85e-deb845c86c8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c13e7a596684498f930cf6b61a2b6124)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:22.033095 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8118c-c82b-4eae-b85e-deb845c86c8b admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:22.300412 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-62f8118c-c82b-4eae-b85e-deb845c86c8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c13e7a596684498f930cf6b61a2b6124'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:22.396301 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-62f8118c-c82b-4eae-b85e-deb845c86c8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c13e7a596684498f930cf6b61a2b6124'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:22.397332 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-62f8118c-c82b-4eae-b85e-deb845c86c8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c13e7a596684498f930cf6b61a2b6124'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:22.414288 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-62f8118c-c82b-4eae-b85e-deb845c86c8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c13e7a596684498f930cf6b61a2b6124'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:22.415156 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-62f8118c-c82b-4eae-b85e-deb845c86c8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c13e7a596684498f930cf6b61a2b6124'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:22.429694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1353/2697] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:20:21 2022] DELETE /identity/v3/users/c13e7a596684498f930cf6b61a2b6124 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:20:22.754684 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf161d-caa9-4f09-8d2d-0402cef48a8d None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:22.768268 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf161d-caa9-4f09-8d2d-0402cef48a8d None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:22.859786 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2195b59c-a7e7-471e-8126-e2aee9e22e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:22.860499 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195b59c-a7e7-471e-8126-e2aee9e22e46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:22.860783 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195b59c-a7e7-471e-8126-e2aee9e22e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:22.861060 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195b59c-a7e7-471e-8126-e2aee9e22e46 admin admin] PATH_INFO: `/v3/projects/1280353c68cf4f8388ea94ba12d1ddcd` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:22.862292 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2195b59c-a7e7-471e-8126-e2aee9e22e46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1280353c68cf4f8388ea94ba12d1ddcd)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:22.864477 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2195b59c-a7e7-471e-8126-e2aee9e22e46 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:23.024405 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2195b59c-a7e7-471e-8126-e2aee9e22e46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1280353c68cf4f8388ea94ba12d1ddcd', 'request_id': 'req-2195b59c-a7e7-471e-8126-e2aee9e22e46'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:23.097049 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2195b59c-a7e7-471e-8126-e2aee9e22e46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1280353c68cf4f8388ea94ba12d1ddcd', 'request_id': 'req-2195b59c-a7e7-471e-8126-e2aee9e22e46'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:23.107580 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2195b59c-a7e7-471e-8126-e2aee9e22e46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1280353c68cf4f8388ea94ba12d1ddcd', 'request_id': 'req-2195b59c-a7e7-471e-8126-e2aee9e22e46'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:23.124591 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2195b59c-a7e7-471e-8126-e2aee9e22e46 admin admin] The token cache is being invalidate because project 1280353c68cf4f8388ea94ba12d1ddcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:20:23.125490 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2195b59c-a7e7-471e-8126-e2aee9e22e46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:23.126952 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2195b59c-a7e7-471e-8126-e2aee9e22e46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:23.133985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1345/2698] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:20:22 2022] DELETE /identity/v3/projects/1280353c68cf4f8388ea94ba12d1ddcd => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:20:23.134452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8118c-c82b-4eae-b85e-deb845c86c8b admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:23.134452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-62f8118c-c82b-4eae-b85e-deb845c86c8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:23.148863 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8118c-c82b-4eae-b85e-deb845c86c8b admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:23.222817 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f73c8d-1f33-40c0-92c3-fb5cecad4288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:23.223606 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f73c8d-1f33-40c0-92c3-fb5cecad4288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:23.223887 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f73c8d-1f33-40c0-92c3-fb5cecad4288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:23.224168 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f73c8d-1f33-40c0-92c3-fb5cecad4288 admin admin] PATH_INFO: `/v3/users/9084137934954072900c7766abde762e` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:23.225513 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f73c8d-1f33-40c0-92c3-fb5cecad4288 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9084137934954072900c7766abde762e)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:23.228341 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f73c8d-1f33-40c0-92c3-fb5cecad4288 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:23.455329 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a8f73c8d-1f33-40c0-92c3-fb5cecad4288 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9084137934954072900c7766abde762e'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:23.530676 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a8f73c8d-1f33-40c0-92c3-fb5cecad4288 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9084137934954072900c7766abde762e'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:23.531538 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a8f73c8d-1f33-40c0-92c3-fb5cecad4288 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9084137934954072900c7766abde762e'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:23.551472 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a8f73c8d-1f33-40c0-92c3-fb5cecad4288 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9084137934954072900c7766abde762e'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:23.552257 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a8f73c8d-1f33-40c0-92c3-fb5cecad4288 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9084137934954072900c7766abde762e'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:23.553807 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:23.568899 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1354/2699] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:20:23 2022] DELETE /identity/v3/users/9084137934954072900c7766abde762e => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:20:23.879486 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2195b59c-a7e7-471e-8126-e2aee9e22e46 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:23.881559 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2195b59c-a7e7-471e-8126-e2aee9e22e46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:23.913350 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2195b59c-a7e7-471e-8126-e2aee9e22e46 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:23.945793 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:24.050770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40222bd3-cc98-4e1d-bd27-30a56b84a2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:24.050770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40222bd3-cc98-4e1d-bd27-30a56b84a2e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:24.050770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40222bd3-cc98-4e1d-bd27-30a56b84a2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:24.050770 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40222bd3-cc98-4e1d-bd27-30a56b84a2e9 admin admin] PATH_INFO: `/v3/projects/4fa014778b1f455180a91fe7c3a6be99` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:24.051971 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40222bd3-cc98-4e1d-bd27-30a56b84a2e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fa014778b1f455180a91fe7c3a6be99)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:24.052331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40222bd3-cc98-4e1d-bd27-30a56b84a2e9 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:24.260288 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-40222bd3-cc98-4e1d-bd27-30a56b84a2e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fa014778b1f455180a91fe7c3a6be99', 'request_id': 'req-40222bd3-cc98-4e1d-bd27-30a56b84a2e9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:24.316750 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-40222bd3-cc98-4e1d-bd27-30a56b84a2e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fa014778b1f455180a91fe7c3a6be99', 'request_id': 'req-40222bd3-cc98-4e1d-bd27-30a56b84a2e9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:24.324092 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-40222bd3-cc98-4e1d-bd27-30a56b84a2e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fa014778b1f455180a91fe7c3a6be99', 'request_id': 'req-40222bd3-cc98-4e1d-bd27-30a56b84a2e9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:24.338405 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-40222bd3-cc98-4e1d-bd27-30a56b84a2e9 admin admin] The token cache is being invalidate because project 4fa014778b1f455180a91fe7c3a6be99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:20:24.338825 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-40222bd3-cc98-4e1d-bd27-30a56b84a2e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:24.344701 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-40222bd3-cc98-4e1d-bd27-30a56b84a2e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:24.344701 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1346/2700] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:20:23 2022] DELETE /identity/v3/projects/4fa014778b1f455180a91fe7c3a6be99 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:20:24.346733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d338a4-f281-439d-bfde-a2532097c15a None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:24.347230 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d338a4-f281-439d-bfde-a2532097c15a None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:24.347515 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d338a4-f281-439d-bfde-a2532097c15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:24.369150 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-a4d338a4-f281-439d-bfde-a2532097c15a None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:24.370640 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-a4d338a4-f281-439d-bfde-a2532097c15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:24.435071 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1355/2701] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:20:24.443652 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115a09b-7b25-4221-8d02-cd976f84b26c None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:24.444125 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115a09b-7b25-4221-8d02-cd976f84b26c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:24.444416 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115a09b-7b25-4221-8d02-cd976f84b26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:24.467375 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-4115a09b-7b25-4221-8d02-cd976f84b26c None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:24.468844 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-4115a09b-7b25-4221-8d02-cd976f84b26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:24.478891 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1347/2702] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:20:24.484500 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d338a4-f281-439d-bfde-a2532097c15a None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:24.493404 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d338a4-f281-439d-bfde-a2532097c15a None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:24.499107 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effcec67-f6c5-4663-b95d-dc30da7e09c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:24.499734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcec67-f6c5-4663-b95d-dc30da7e09c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:24.500020 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcec67-f6c5-4663-b95d-dc30da7e09c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:24.500297 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcec67-f6c5-4663-b95d-dc30da7e09c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:24.501530 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effcec67-f6c5-4663-b95d-dc30da7e09c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:20:24.502110 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effcec67-f6c5-4663-b95d-dc30da7e09c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:24.503583 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effcec67-f6c5-4663-b95d-dc30da7e09c9 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:24.530054 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1356/2703] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 05:20:24.531302 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4115a09b-7b25-4221-8d02-cd976f84b26c None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:24.541142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4115a09b-7b25-4221-8d02-cd976f84b26c None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:24.547029 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7806ff5-6fe0-42a0-85f2-3b441dd066d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:24.547982 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7806ff5-6fe0-42a0-85f2-3b441dd066d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:24.548446 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7806ff5-6fe0-42a0-85f2-3b441dd066d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:24.548926 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7806ff5-6fe0-42a0-85f2-3b441dd066d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:24.549783 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7806ff5-6fe0-42a0-85f2-3b441dd066d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:24.551949 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7806ff5-6fe0-42a0-85f2-3b441dd066d4 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:24.642333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1348/2704] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:20:24 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:20:24.646439 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effcec67-f6c5-4663-b95d-dc30da7e09c9 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:24.656453 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effcec67-f6c5-4663-b95d-dc30da7e09c9 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:24.720360 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee98c3-6726-4eab-b0a6-7e01903d2e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:24.721209 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee98c3-6726-4eab-b0a6-7e01903d2e41 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:24.721719 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee98c3-6726-4eab-b0a6-7e01903d2e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:24.722057 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee98c3-6726-4eab-b0a6-7e01903d2e41 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:24.722835 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee98c3-6726-4eab-b0a6-7e01903d2e41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:24.725073 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee98c3-6726-4eab-b0a6-7e01903d2e41 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:24.839532 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1357/2705] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:20:24 2022] POST /identity/v3/users => generated 287 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:20:24.843352 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7806ff5-6fe0-42a0-85f2-3b441dd066d4 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:24.854622 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7806ff5-6fe0-42a0-85f2-3b441dd066d4 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:24.862815 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818594d4-250a-47d5-9305-ca52913246b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:24.863826 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818594d4-250a-47d5-9305-ca52913246b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:24.864319 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818594d4-250a-47d5-9305-ca52913246b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:24.864944 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818594d4-250a-47d5-9305-ca52913246b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:24.865978 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818594d4-250a-47d5-9305-ca52913246b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:24.868055 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818594d4-250a-47d5-9305-ca52913246b8 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:24.880410 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1349/2706] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:20:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:20:24.885001 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee98c3-6726-4eab-b0a6-7e01903d2e41 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:24.900424 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee98c3-6726-4eab-b0a6-7e01903d2e41 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:24.903122 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72344c0d-49c2-4eca-9e41-0a14e577cd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:24.903758 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72344c0d-49c2-4eca-9e41-0a14e577cd44 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:24.904042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72344c0d-49c2-4eca-9e41-0a14e577cd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:24.904318 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72344c0d-49c2-4eca-9e41-0a14e577cd44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:24.905212 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72344c0d-49c2-4eca-9e41-0a14e577cd44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:24.907548 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72344c0d-49c2-4eca-9e41-0a14e577cd44 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:24.920012 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1358/2707] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:20:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:20:24.924647 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818594d4-250a-47d5-9305-ca52913246b8 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:24.937828 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818594d4-250a-47d5-9305-ca52913246b8 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:24.943625 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7531ba2b-d52b-464f-8db8-515635b27ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:24.944710 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531ba2b-d52b-464f-8db8-515635b27ef8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:24.945172 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531ba2b-d52b-464f-8db8-515635b27ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:24.945623 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531ba2b-d52b-464f-8db8-515635b27ef8 admin admin] PATH_INFO: `/v3/projects/d365a7057d45426ea74a671cdece5310/users/486941d8fd1e49deac5f1de878b3e281/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:24.972834 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7531ba2b-d52b-464f-8db8-515635b27ef8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d365a7057d45426ea74a671cdece5310, user_id=486941d8fd1e49deac5f1de878b3e281, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:24.975841 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7531ba2b-d52b-464f-8db8-515635b27ef8 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:25.071684 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1350/2708] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:20:24 2022] PUT /identity/v3/projects/d365a7057d45426ea74a671cdece5310/users/486941d8fd1e49deac5f1de878b3e281/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:20:25.078085 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c01be-f9f2-489d-ac79-ec34035906b2 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:25.078380 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c01be-f9f2-489d-ac79-ec34035906b2 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:25.078695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c01be-f9f2-489d-ac79-ec34035906b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:25.121793 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-367c01be-f9f2-489d-ac79-ec34035906b2 None None] MFA Rules not processed for user `486941d8fd1e49deac5f1de878b3e281`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:25.123470 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-367c01be-f9f2-489d-ac79-ec34035906b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:25.182094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:25.215537 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1359/2709] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:20:25.223573 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7531ba2b-d52b-464f-8db8-515635b27ef8 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:25.277332 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:25.291484 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7531ba2b-d52b-464f-8db8-515635b27ef8 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:25.296542 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13478362-fd7c-48c3-bf26-7d057a542725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:25.297183 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13478362-fd7c-48c3-bf26-7d057a542725 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:25.297467 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13478362-fd7c-48c3-bf26-7d057a542725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:25.297772 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13478362-fd7c-48c3-bf26-7d057a542725 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:25.298486 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13478362-fd7c-48c3-bf26-7d057a542725 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:25.304036 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13478362-fd7c-48c3-bf26-7d057a542725 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:25.405892 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1351/2710] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:20:25 2022] POST /identity/v3/projects => generated 373 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:20:25.409897 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c01be-f9f2-489d-ac79-ec34035906b2 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:25.426744 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c01be-f9f2-489d-ac79-ec34035906b2 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:25.556640 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592cff18-b433-49a5-842d-15ecca64b3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:25.557897 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592cff18-b433-49a5-842d-15ecca64b3c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:25.558204 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592cff18-b433-49a5-842d-15ecca64b3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:25.558619 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592cff18-b433-49a5-842d-15ecca64b3c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:25.559356 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592cff18-b433-49a5-842d-15ecca64b3c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:25.562874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592cff18-b433-49a5-842d-15ecca64b3c6 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:25.655610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1360/2711] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:20:25 2022] POST /identity/v3/users => generated 293 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:20:25.657395 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13478362-fd7c-48c3-bf26-7d057a542725 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:25.671529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13478362-fd7c-48c3-bf26-7d057a542725 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:25.677877 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c909b-bedd-406e-9d34-f726f6a1805c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:25.678715 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c909b-bedd-406e-9d34-f726f6a1805c admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:25.679137 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c909b-bedd-406e-9d34-f726f6a1805c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:25.679794 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c909b-bedd-406e-9d34-f726f6a1805c admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:25.680856 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c909b-bedd-406e-9d34-f726f6a1805c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:25.683886 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c909b-bedd-406e-9d34-f726f6a1805c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:25.703735 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1352/2712] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:20:25.708176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592cff18-b433-49a5-842d-15ecca64b3c6 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:25.731206 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592cff18-b433-49a5-842d-15ecca64b3c6 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:25.737950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207569f-001c-431c-8ca7-ea619bb3e9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:25.738657 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207569f-001c-431c-8ca7-ea619bb3e9f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:25.739086 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207569f-001c-431c-8ca7-ea619bb3e9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:25.739372 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207569f-001c-431c-8ca7-ea619bb3e9f9 admin admin] PATH_INFO: `/v3/projects/00d59a2a28b64f00a3bc8d27ccd9ac0a/users/43cb895aebfd4cf2a079108fd29aa8c8/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:25.759387 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207569f-001c-431c-8ca7-ea619bb3e9f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00d59a2a28b64f00a3bc8d27ccd9ac0a, user_id=43cb895aebfd4cf2a079108fd29aa8c8, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:25.762353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207569f-001c-431c-8ca7-ea619bb3e9f9 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:25.783741 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0538ef1-ae1e-4c6a-a21e-c43abdbac857 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:25.783741 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0538ef1-ae1e-4c6a-a21e-c43abdbac857 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:25.784381 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0538ef1-ae1e-4c6a-a21e-c43abdbac857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:25.817654 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-f0538ef1-ae1e-4c6a-a21e-c43abdbac857 None None] MFA Rules not processed for user `f2d6f433ebaa40d2bc5dfaba438b676f`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:25.819076 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f0538ef1-ae1e-4c6a-a21e-c43abdbac857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:25.836461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1361/2713] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:20:25 2022] PUT /identity/v3/projects/00d59a2a28b64f00a3bc8d27ccd9ac0a/users/43cb895aebfd4cf2a079108fd29aa8c8/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:20:25.840546 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207569f-001c-431c-8ca7-ea619bb3e9f9 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:25.919466 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207569f-001c-431c-8ca7-ea619bb3e9f9 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:25.921366 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1353/2714] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:25 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:20:25.925271 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd1909-6e98-425c-9b49-30c881c0989e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:25.925978 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd1909-6e98-425c-9b49-30c881c0989e admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:25.926987 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd1909-6e98-425c-9b49-30c881c0989e admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:25.927286 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd1909-6e98-425c-9b49-30c881c0989e admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:25.927986 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd1909-6e98-425c-9b49-30c881c0989e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:25.938256 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0538ef1-ae1e-4c6a-a21e-c43abdbac857 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:25.939035 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd1909-6e98-425c-9b49-30c881c0989e admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:25.940449 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f0538ef1-ae1e-4c6a-a21e-c43abdbac857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:25.951452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1362/2715] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:20:25.960070 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd1909-6e98-425c-9b49-30c881c0989e admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:25.971214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd1909-6e98-425c-9b49-30c881c0989e admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:25.977333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d6fe1-d156-4627-a4e0-775283ed7f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:25.978054 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d6fe1-d156-4627-a4e0-775283ed7f95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:25.978342 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d6fe1-d156-4627-a4e0-775283ed7f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:25.978896 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d6fe1-d156-4627-a4e0-775283ed7f95 admin admin] PATH_INFO: `/v3/projects/00d59a2a28b64f00a3bc8d27ccd9ac0a/users/43cb895aebfd4cf2a079108fd29aa8c8/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:25.980963 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d6fe1-d156-4627-a4e0-775283ed7f95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00d59a2a28b64f00a3bc8d27ccd9ac0a, user_id=43cb895aebfd4cf2a079108fd29aa8c8, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:25.984225 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d6fe1-d156-4627-a4e0-775283ed7f95 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:25.997778 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1363/2716] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:20:25 2022] PUT /identity/v3/projects/00d59a2a28b64f00a3bc8d27ccd9ac0a/users/43cb895aebfd4cf2a079108fd29aa8c8/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:20:26.014028 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c18021-68e1-4455-8bd6-646c9ab67f8c None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:26.014410 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c18021-68e1-4455-8bd6-646c9ab67f8c None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:26.014869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c18021-68e1-4455-8bd6-646c9ab67f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:26.039392 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0538ef1-ae1e-4c6a-a21e-c43abdbac857 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:26.058784 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-25c18021-68e1-4455-8bd6-646c9ab67f8c None None] MFA Rules not processed for user `43cb895aebfd4cf2a079108fd29aa8c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:26.060649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-25c18021-68e1-4455-8bd6-646c9ab67f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:26.095681 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:26.107986 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:26.134451 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e81d665-c0a9-4505-a8bf-c22d8712a39b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:26.135507 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81d665-c0a9-4505-a8bf-c22d8712a39b service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:26.135991 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81d665-c0a9-4505-a8bf-c22d8712a39b service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:26.136449 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81d665-c0a9-4505-a8bf-c22d8712a39b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:26.138604 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-5e81d665-c0a9-4505-a8bf-c22d8712a39b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:26.162711 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1364/2717] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:20:26.176024 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c18021-68e1-4455-8bd6-646c9ab67f8c None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:26.177822 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-25c18021-68e1-4455-8bd6-646c9ab67f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:26.211816 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e81d665-c0a9-4505-a8bf-c22d8712a39b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:26.213848 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e81d665-c0a9-4505-a8bf-c22d8712a39b service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:26.226368 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1354/2718] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:20:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:20:26.256492 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c18021-68e1-4455-8bd6-646c9ab67f8c None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:26.260270 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a4aa9-ff70-4aae-acf4-606c078cd9f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:26.261129 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a4aa9-ff70-4aae-acf4-606c078cd9f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:26.261474 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a4aa9-ff70-4aae-acf4-606c078cd9f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:26.261946 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a4aa9-ff70-4aae-acf4-606c078cd9f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:26.272570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2a4aa9-ff70-4aae-acf4-606c078cd9f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:26.275740 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2a4aa9-ff70-4aae-acf4-606c078cd9f8 service cinder] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:26.286212 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1365/2719] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:20:26 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:20:26.507782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e81d665-c0a9-4505-a8bf-c22d8712a39b service nova] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:26.517252 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e81d665-c0a9-4505-a8bf-c22d8712a39b service nova] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:26.520889 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd331a-0db6-41fb-8a84-36a6df603aa3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:26.521498 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd331a-0db6-41fb-8a84-36a6df603aa3 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:26.521779 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd331a-0db6-41fb-8a84-36a6df603aa3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:26.522052 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd331a-0db6-41fb-8a84-36a6df603aa3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:26.580746 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd331a-0db6-41fb-8a84-36a6df603aa3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:26.582521 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd331a-0db6-41fb-8a84-36a6df603aa3 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:26.592233 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1355/2720] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:20:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:20:36.281600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=45 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:36.588091 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=63 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:39.720501 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a4aa9-ff70-4aae-acf4-606c078cd9f8 service cinder] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:39.721980 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-aa2a4aa9-ff70-4aae-acf4-606c078cd9f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:39.780650 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a4aa9-ff70-4aae-acf4-606c078cd9f8 service cinder] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:39.784257 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dbb0d4-6d28-4315-9ddc-b5c0c8d5ad4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:39.784860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbb0d4-6d28-4315-9ddc-b5c0c8d5ad4c service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:39.785297 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbb0d4-6d28-4315-9ddc-b5c0c8d5ad4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:39.785465 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbb0d4-6d28-4315-9ddc-b5c0c8d5ad4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:39.787213 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-32dbb0d4-6d28-4315-9ddc-b5c0c8d5ad4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:39.849415 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dbb0d4-6d28-4315-9ddc-b5c0c8d5ad4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:39.850883 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dbb0d4-6d28-4315-9ddc-b5c0c8d5ad4c service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:39.860496 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1366/2721] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:20:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:20:43.765946 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd331a-0db6-41fb-8a84-36a6df603aa3 service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:43.767563 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-cfdd331a-0db6-41fb-8a84-36a6df603aa3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:43.783443 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd331a-0db6-41fb-8a84-36a6df603aa3 service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:43.788740 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91c111-1a30-4f6a-8421-72e39a9aca92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:43.789400 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91c111-1a30-4f6a-8421-72e39a9aca92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:43.789680 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91c111-1a30-4f6a-8421-72e39a9aca92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:43.792467 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91c111-1a30-4f6a-8421-72e39a9aca92 admin admin] PATH_INFO: `/v3/users/747a608061504d7693f0f99ecdfa476e` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:43.792467 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c91c111-1a30-4f6a-8421-72e39a9aca92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=747a608061504d7693f0f99ecdfa476e)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:43.793438 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c91c111-1a30-4f6a-8421-72e39a9aca92 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:44.042832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4c91c111-1a30-4f6a-8421-72e39a9aca92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '747a608061504d7693f0f99ecdfa476e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:44.119853 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4c91c111-1a30-4f6a-8421-72e39a9aca92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '747a608061504d7693f0f99ecdfa476e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:44.120442 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4c91c111-1a30-4f6a-8421-72e39a9aca92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '747a608061504d7693f0f99ecdfa476e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:44.134685 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4c91c111-1a30-4f6a-8421-72e39a9aca92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '747a608061504d7693f0f99ecdfa476e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:44.135429 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4c91c111-1a30-4f6a-8421-72e39a9aca92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '747a608061504d7693f0f99ecdfa476e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:44.148286 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1356/2722] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:20:43 2022] DELETE /identity/v3/users/747a608061504d7693f0f99ecdfa476e => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:20:44.424992 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dbb0d4-6d28-4315-9ddc-b5c0c8d5ad4c service neutron] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:44.435408 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dbb0d4-6d28-4315-9ddc-b5c0c8d5ad4c service neutron] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:44.495736 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7944037d-f9d8-482b-bc01-feeaf7bc3adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:44.496375 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944037d-f9d8-482b-bc01-feeaf7bc3adb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:44.496659 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944037d-f9d8-482b-bc01-feeaf7bc3adb admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:44.496942 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944037d-f9d8-482b-bc01-feeaf7bc3adb admin admin] PATH_INFO: `/v3/projects/050592077e0b4d26a1555d2daa7f460d` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:44.498128 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7944037d-f9d8-482b-bc01-feeaf7bc3adb admin admin] RBAC: Authorizing `identity:delete_project(project_id=050592077e0b4d26a1555d2daa7f460d)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:44.500082 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7944037d-f9d8-482b-bc01-feeaf7bc3adb admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:44.614024 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-7944037d-f9d8-482b-bc01-feeaf7bc3adb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '050592077e0b4d26a1555d2daa7f460d', 'request_id': 'req-7944037d-f9d8-482b-bc01-feeaf7bc3adb'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:44.615962 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:44.621086 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-7944037d-f9d8-482b-bc01-feeaf7bc3adb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '050592077e0b4d26a1555d2daa7f460d', 'request_id': 'req-7944037d-f9d8-482b-bc01-feeaf7bc3adb'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:44.653753 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d83b2-4469-49db-85af-afb5e8d8836b None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:44.654043 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d83b2-4469-49db-85af-afb5e8d8836b None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:44.654363 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d83b2-4469-49db-85af-afb5e8d8836b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:44.674027 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-e50d83b2-4469-49db-85af-afb5e8d8836b None None] MFA Rules not processed for user `ac3954ab39014ef3819111a6fe8142b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:44.675656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e50d83b2-4469-49db-85af-afb5e8d8836b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:44.701055 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-7944037d-f9d8-482b-bc01-feeaf7bc3adb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '050592077e0b4d26a1555d2daa7f460d', 'request_id': 'req-7944037d-f9d8-482b-bc01-feeaf7bc3adb'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:44.713554 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-7944037d-f9d8-482b-bc01-feeaf7bc3adb admin admin] The token cache is being invalidate because project 050592077e0b4d26a1555d2daa7f460d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:20:44.714172 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-7944037d-f9d8-482b-bc01-feeaf7bc3adb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:44.715164 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-7944037d-f9d8-482b-bc01-feeaf7bc3adb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:20:44.716943 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1367/2723] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:20:44 2022] DELETE /identity/v3/projects/050592077e0b4d26a1555d2daa7f460d => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:20:44.722140 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a9f46-7fd2-488e-8779-a56fbf011952 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:44.722428 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a9f46-7fd2-488e-8779-a56fbf011952 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:44.722705 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a9f46-7fd2-488e-8779-a56fbf011952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:44.733769 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1357/2724] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:20:44.742959 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-4c3a9f46-7fd2-488e-8779-a56fbf011952 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:44.744319 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-4c3a9f46-7fd2-488e-8779-a56fbf011952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:44.807385 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1368/2725] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:20:44.815391 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39be09-5581-4a46-bc17-55300be33b8b None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:44.815863 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39be09-5581-4a46-bc17-55300be33b8b None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:44.815964 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39be09-5581-4a46-bc17-55300be33b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:44.839975 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-9d39be09-5581-4a46-bc17-55300be33b8b None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:44.841252 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9d39be09-5581-4a46-bc17-55300be33b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:44.849476 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1358/2726] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:20:44.853794 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a9f46-7fd2-488e-8779-a56fbf011952 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:44.867544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a9f46-7fd2-488e-8779-a56fbf011952 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:44.872427 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ac52f9-26ef-4ff8-adc8-3e9c191929f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:44.873105 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac52f9-26ef-4ff8-adc8-3e9c191929f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:44.873384 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac52f9-26ef-4ff8-adc8-3e9c191929f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:44.873664 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac52f9-26ef-4ff8-adc8-3e9c191929f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:44.874365 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ac52f9-26ef-4ff8-adc8-3e9c191929f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:20:44.874673 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ac52f9-26ef-4ff8-adc8-3e9c191929f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:44.876401 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ac52f9-26ef-4ff8-adc8-3e9c191929f8 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:44.897733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1369/2727] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:20:44.902216 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d39be09-5581-4a46-bc17-55300be33b8b None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:44.912412 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d39be09-5581-4a46-bc17-55300be33b8b None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:44.920259 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4caaa-5916-470c-a204-e5b1e765f2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:44.920259 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4caaa-5916-470c-a204-e5b1e765f2a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:44.920685 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4caaa-5916-470c-a204-e5b1e765f2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:44.920787 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4caaa-5916-470c-a204-e5b1e765f2a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:44.921462 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e4caaa-5916-470c-a204-e5b1e765f2a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:44.923519 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e4caaa-5916-470c-a204-e5b1e765f2a5 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:45.045218 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1359/2728] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:20:44 2022] POST /identity/v3/projects => generated 385 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:20:45.051161 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ac52f9-26ef-4ff8-adc8-3e9c191929f8 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:45.062733 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ac52f9-26ef-4ff8-adc8-3e9c191929f8 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:45.085841 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4caaa-5916-470c-a204-e5b1e765f2a5 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:45.087580 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d5e4caaa-5916-470c-a204-e5b1e765f2a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:45.143697 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8becfcf9-7f64-413e-a41b-26e415af9e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:45.144316 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8becfcf9-7f64-413e-a41b-26e415af9e17 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:45.144605 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8becfcf9-7f64-413e-a41b-26e415af9e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:45.144878 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8becfcf9-7f64-413e-a41b-26e415af9e17 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:45.145642 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8becfcf9-7f64-413e-a41b-26e415af9e17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:45.147891 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8becfcf9-7f64-413e-a41b-26e415af9e17 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:45.156672 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:45.161887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4caaa-5916-470c-a204-e5b1e765f2a5 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:45.165963 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6110a1-af3e-4520-9c7b-309dc5d6a317 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:45.166589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6110a1-af3e-4520-9c7b-309dc5d6a317 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:45.166901 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6110a1-af3e-4520-9c7b-309dc5d6a317 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:45.167191 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6110a1-af3e-4520-9c7b-309dc5d6a317 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:45.169056 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7b6110a1-af3e-4520-9c7b-309dc5d6a317 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:45.180335 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6110a1-af3e-4520-9c7b-309dc5d6a317 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:45.182077 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6110a1-af3e-4520-9c7b-309dc5d6a317 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:45.191264 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1360/2729] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:20:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:20:45.275715 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1370/2730] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:20:45 2022] POST /identity/v3/users => generated 293 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:20:45.282098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6110a1-af3e-4520-9c7b-309dc5d6a317 service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:45.298821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6110a1-af3e-4520-9c7b-309dc5d6a317 service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:45.305552 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca4b2f-7163-4806-86c0-ad2fce2d4b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:45.310977 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca4b2f-7163-4806-86c0-ad2fce2d4b90 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:45.310977 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca4b2f-7163-4806-86c0-ad2fce2d4b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:45.310977 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca4b2f-7163-4806-86c0-ad2fce2d4b90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:45.310977 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca4b2f-7163-4806-86c0-ad2fce2d4b90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:45.310977 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca4b2f-7163-4806-86c0-ad2fce2d4b90 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:45.324979 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1361/2731] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:20:45.326211 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8becfcf9-7f64-413e-a41b-26e415af9e17 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:45.337057 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8becfcf9-7f64-413e-a41b-26e415af9e17 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:45.341772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356fcec8-aa59-4022-b514-3e679eedc7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:45.342390 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356fcec8-aa59-4022-b514-3e679eedc7c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:45.342671 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356fcec8-aa59-4022-b514-3e679eedc7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:45.342981 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356fcec8-aa59-4022-b514-3e679eedc7c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:45.343681 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356fcec8-aa59-4022-b514-3e679eedc7c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:45.345378 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356fcec8-aa59-4022-b514-3e679eedc7c0 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:45.357120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1371/2732] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:20:45.365330 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca4b2f-7163-4806-86c0-ad2fce2d4b90 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:45.375522 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca4b2f-7163-4806-86c0-ad2fce2d4b90 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:45.381514 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797b765-1b31-4771-a48c-25ce75c6fc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:45.382217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797b765-1b31-4771-a48c-25ce75c6fc22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:45.382505 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797b765-1b31-4771-a48c-25ce75c6fc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:45.382793 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797b765-1b31-4771-a48c-25ce75c6fc22 admin admin] PATH_INFO: `/v3/projects/041a553912c64de981d9f87b551c190f/users/729a6d00e1664a6298088e52de9999f5/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:45.404673 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b797b765-1b31-4771-a48c-25ce75c6fc22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=041a553912c64de981d9f87b551c190f, user_id=729a6d00e1664a6298088e52de9999f5, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:45.408066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b797b765-1b31-4771-a48c-25ce75c6fc22 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:45.508444 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1362/2733] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:20:45 2022] PUT /identity/v3/projects/041a553912c64de981d9f87b551c190f/users/729a6d00e1664a6298088e52de9999f5/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:20:45.509769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58923ad9-4a09-4f72-a67c-beb9a363fda6 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:45.510487 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58923ad9-4a09-4f72-a67c-beb9a363fda6 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:45.510487 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58923ad9-4a09-4f72-a67c-beb9a363fda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:45.541986 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-58923ad9-4a09-4f72-a67c-beb9a363fda6 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:45.542747 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-58923ad9-4a09-4f72-a67c-beb9a363fda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:45.594034 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:45.612695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1372/2734] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:20:45.622299 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797b765-1b31-4771-a48c-25ce75c6fc22 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:45.692351 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797b765-1b31-4771-a48c-25ce75c6fc22 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:45.696854 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97cc9d4-5275-487f-9e74-940f000ee817 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:45.697478 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cc9d4-5275-487f-9e74-940f000ee817 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:45.699229 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cc9d4-5275-487f-9e74-940f000ee817 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:45.699515 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cc9d4-5275-487f-9e74-940f000ee817 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:45.716668 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97cc9d4-5275-487f-9e74-940f000ee817 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:45.716668 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97cc9d4-5275-487f-9e74-940f000ee817 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:45.731709 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1363/2735] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:20:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:20:53.143601 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c76a04-4a3d-4845-a5c4-edf94dbfa7ab None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:53.144150 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c76a04-4a3d-4845-a5c4-edf94dbfa7ab None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:53.144273 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c76a04-4a3d-4845-a5c4-edf94dbfa7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:53.186142 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-a3c76a04-4a3d-4845-a5c4-edf94dbfa7ab None None] MFA Rules not processed for user `729a6d00e1664a6298088e52de9999f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:53.187561 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-a3c76a04-4a3d-4845-a5c4-edf94dbfa7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:53.281188 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1373/2736] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:20:53.295849 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97cc9d4-5275-487f-9e74-940f000ee817 service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:53.297913 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a97cc9d4-5275-487f-9e74-940f000ee817 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:53.450787 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:20:53.471611 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97cc9d4-5275-487f-9e74-940f000ee817 service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:53.478774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a19048-fff0-403a-91a8-14629a129c75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:53.479495 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a19048-fff0-403a-91a8-14629a129c75 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:53.479851 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a19048-fff0-403a-91a8-14629a129c75 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:53.480219 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a19048-fff0-403a-91a8-14629a129c75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:53.489351 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a19048-fff0-403a-91a8-14629a129c75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:53.491104 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a19048-fff0-403a-91a8-14629a129c75 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:53.500950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1364/2737] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:20:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:20:56.022976 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c6197-32f7-4ca3-903c-13f5b1cd0828 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:56.023382 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c6197-32f7-4ca3-903c-13f5b1cd0828 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:56.023579 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c6197-32f7-4ca3-903c-13f5b1cd0828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:56.042651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-785c6197-32f7-4ca3-903c-13f5b1cd0828 None None] MFA Rules not processed for user `729a6d00e1664a6298088e52de9999f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:20:56.043968 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-785c6197-32f7-4ca3-903c-13f5b1cd0828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:56.051908 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1374/2738] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:20:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:20:58.558870 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a19048-fff0-403a-91a8-14629a129c75 service nova] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:20:58.560565 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e3a19048-fff0-403a-91a8-14629a129c75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:58.664251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a19048-fff0-403a-91a8-14629a129c75 service nova] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:20:58.665147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e41b503-2201-4a02-afb2-844114ca3f03 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59846d88f4314c2382aa5cb8bae5053b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:20:58.665756 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41b503-2201-4a02-afb2-844114ca3f03 service placement] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:20:58.666037 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41b503-2201-4a02-afb2-844114ca3f03 service placement] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:20:58.666656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41b503-2201-4a02-afb2-844114ca3f03 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:20:58.668056 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2e41b503-2201-4a02-afb2-844114ca3f03 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:20:58.680230 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e41b503-2201-4a02-afb2-844114ca3f03 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:20:58.681927 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e41b503-2201-4a02-afb2-844114ca3f03 service placement] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:20:58.693227 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1365/2739] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:20:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:21:06.037688 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=53 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:21:06.773863 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c6197-32f7-4ca3-903c-13f5b1cd0828 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:06.793262 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c6197-32f7-4ca3-903c-13f5b1cd0828 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:06.796723 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026df73-0ccd-4423-b2c2-f9e556e16a35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:06.797324 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026df73-0ccd-4423-b2c2-f9e556e16a35 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:06.797601 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026df73-0ccd-4423-b2c2-f9e556e16a35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:06.797873 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026df73-0ccd-4423-b2c2-f9e556e16a35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:06.799662 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-8026df73-0ccd-4423-b2c2-f9e556e16a35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:21:06.872459 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026df73-0ccd-4423-b2c2-f9e556e16a35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:06.873950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026df73-0ccd-4423-b2c2-f9e556e16a35 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:06.889277 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1375/2740] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:21:06 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:21:07.156687 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e41b503-2201-4a02-afb2-844114ca3f03 service placement] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:07.158184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2e41b503-2201-4a02-afb2-844114ca3f03 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:21:07.183755 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e41b503-2201-4a02-afb2-844114ca3f03 service placement] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:07.187491 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb855fe-22f4-4ab0-9c35-b8dd216094bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:07.188105 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb855fe-22f4-4ab0-9c35-b8dd216094bc service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:07.188392 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb855fe-22f4-4ab0-9c35-b8dd216094bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:07.188670 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb855fe-22f4-4ab0-9c35-b8dd216094bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:07.190554 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2eb855fe-22f4-4ab0-9c35-b8dd216094bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:21:07.264309 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb855fe-22f4-4ab0-9c35-b8dd216094bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:07.266105 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb855fe-22f4-4ab0-9c35-b8dd216094bc service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:07.276706 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1366/2741] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:21:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:21:12.872949 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026df73-0ccd-4423-b2c2-f9e556e16a35 service neutron] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:12.874440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-8026df73-0ccd-4423-b2c2-f9e556e16a35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:21:12.939308 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026df73-0ccd-4423-b2c2-f9e556e16a35 service neutron] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:12.946291 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6c8f3-20ad-41af-ac61-81e3b1e19ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:12.947635 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6c8f3-20ad-41af-ac61-81e3b1e19ed4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:12.948221 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6c8f3-20ad-41af-ac61-81e3b1e19ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:12.952974 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6c8f3-20ad-41af-ac61-81e3b1e19ed4 admin admin] PATH_INFO: `/v3/users/486941d8fd1e49deac5f1de878b3e281` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:12.952974 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6c8f3-20ad-41af-ac61-81e3b1e19ed4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=486941d8fd1e49deac5f1de878b3e281)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:12.952974 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6c8f3-20ad-41af-ac61-81e3b1e19ed4 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:13.169360 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-f1c6c8f3-20ad-41af-ac61-81e3b1e19ed4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '486941d8fd1e49deac5f1de878b3e281'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:13.261181 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-f1c6c8f3-20ad-41af-ac61-81e3b1e19ed4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '486941d8fd1e49deac5f1de878b3e281'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:13.261848 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-f1c6c8f3-20ad-41af-ac61-81e3b1e19ed4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '486941d8fd1e49deac5f1de878b3e281'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:13.281721 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-f1c6c8f3-20ad-41af-ac61-81e3b1e19ed4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '486941d8fd1e49deac5f1de878b3e281'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:13.282627 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-f1c6c8f3-20ad-41af-ac61-81e3b1e19ed4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '486941d8fd1e49deac5f1de878b3e281'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:13.296290 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1376/2742] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:21:12 2022] DELETE /identity/v3/users/486941d8fd1e49deac5f1de878b3e281 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:21:13.301038 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d217e59-ef52-458d-8110-2f41175b90e3 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:13.301919 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d217e59-ef52-458d-8110-2f41175b90e3 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:13.302200 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d217e59-ef52-458d-8110-2f41175b90e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:13.334875 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-2d217e59-ef52-458d-8110-2f41175b90e3 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:21:13.336356 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2d217e59-ef52-458d-8110-2f41175b90e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:21:13.362047 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:21:13.399280 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1367/2743] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:21:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:21:13.410768 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6c8f3-20ad-41af-ac61-81e3b1e19ed4 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:13.412311 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-f1c6c8f3-20ad-41af-ac61-81e3b1e19ed4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:21:13.450908 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:21:13.480406 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6c8f3-20ad-41af-ac61-81e3b1e19ed4 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:13.484651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bcf264-c9ac-4131-99da-3dfd263baf53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:13.485258 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcf264-c9ac-4131-99da-3dfd263baf53 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:13.485696 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcf264-c9ac-4131-99da-3dfd263baf53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:13.485977 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcf264-c9ac-4131-99da-3dfd263baf53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:13.495761 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bcf264-c9ac-4131-99da-3dfd263baf53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:13.497225 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bcf264-c9ac-4131-99da-3dfd263baf53 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:13.510039 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1377/2744] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:21:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:21:13.986523 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d217e59-ef52-458d-8110-2f41175b90e3 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:13.986523 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2d217e59-ef52-458d-8110-2f41175b90e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:21:14.007074 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d217e59-ef52-458d-8110-2f41175b90e3 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:14.019998 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b88512-bbc3-4f0c-936a-a82e76f83734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:14.020725 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b88512-bbc3-4f0c-936a-a82e76f83734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:14.021190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b88512-bbc3-4f0c-936a-a82e76f83734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:14.021642 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b88512-bbc3-4f0c-936a-a82e76f83734 admin admin] PATH_INFO: `/v3/projects/d365a7057d45426ea74a671cdece5310` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:14.023632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b88512-bbc3-4f0c-936a-a82e76f83734 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d365a7057d45426ea74a671cdece5310)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:14.025887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b88512-bbc3-4f0c-936a-a82e76f83734 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:14.082469 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bcf264-c9ac-4131-99da-3dfd263baf53 service neutron] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:14.084226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-e9bcf264-c9ac-4131-99da-3dfd263baf53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:21:14.099450 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bcf264-c9ac-4131-99da-3dfd263baf53 service neutron] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:14.103274 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e8471-b66b-4384-bdec-4ec211e113e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:14.104081 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e8471-b66b-4384-bdec-4ec211e113e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:14.104378 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e8471-b66b-4384-bdec-4ec211e113e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:14.104654 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e8471-b66b-4384-bdec-4ec211e113e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:14.106593 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-993e8471-b66b-4384-bdec-4ec211e113e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:21:14.119268 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993e8471-b66b-4384-bdec-4ec211e113e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:14.120937 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993e8471-b66b-4384-bdec-4ec211e113e0 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:14.131906 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1378/2745] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:21:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:21:14.196822 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b0b88512-bbc3-4f0c-936a-a82e76f83734 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd365a7057d45426ea74a671cdece5310', 'request_id': 'req-b0b88512-bbc3-4f0c-936a-a82e76f83734'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:14.267084 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b0b88512-bbc3-4f0c-936a-a82e76f83734 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd365a7057d45426ea74a671cdece5310', 'request_id': 'req-b0b88512-bbc3-4f0c-936a-a82e76f83734'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:14.278458 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b0b88512-bbc3-4f0c-936a-a82e76f83734 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd365a7057d45426ea74a671cdece5310', 'request_id': 'req-b0b88512-bbc3-4f0c-936a-a82e76f83734'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:14.295574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b0b88512-bbc3-4f0c-936a-a82e76f83734 admin admin] The token cache is being invalidate because project d365a7057d45426ea74a671cdece5310 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:21:14.296821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b0b88512-bbc3-4f0c-936a-a82e76f83734 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:14.296821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b0b88512-bbc3-4f0c-936a-a82e76f83734 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:14.299133 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1368/2746] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:21:13 2022] DELETE /identity/v3/projects/d365a7057d45426ea74a671cdece5310 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:21:14.302999 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e8471-b66b-4384-bdec-4ec211e113e0 service neutron] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:14.313082 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e8471-b66b-4384-bdec-4ec211e113e0 service neutron] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:14.375311 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df9f57-cca5-42d9-93f0-11016ca2b208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:14.375968 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df9f57-cca5-42d9-93f0-11016ca2b208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:14.376257 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df9f57-cca5-42d9-93f0-11016ca2b208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:14.376537 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df9f57-cca5-42d9-93f0-11016ca2b208 admin admin] PATH_INFO: `/v3/users/43cb895aebfd4cf2a079108fd29aa8c8` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:14.381673 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df9f57-cca5-42d9-93f0-11016ca2b208 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43cb895aebfd4cf2a079108fd29aa8c8)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:14.383886 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df9f57-cca5-42d9-93f0-11016ca2b208 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:14.601856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-c4df9f57-cca5-42d9-93f0-11016ca2b208 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43cb895aebfd4cf2a079108fd29aa8c8'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:14.673023 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-c4df9f57-cca5-42d9-93f0-11016ca2b208 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43cb895aebfd4cf2a079108fd29aa8c8'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:14.673595 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-c4df9f57-cca5-42d9-93f0-11016ca2b208 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43cb895aebfd4cf2a079108fd29aa8c8'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:14.687490 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-c4df9f57-cca5-42d9-93f0-11016ca2b208 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43cb895aebfd4cf2a079108fd29aa8c8'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:14.688216 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-c4df9f57-cca5-42d9-93f0-11016ca2b208 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43cb895aebfd4cf2a079108fd29aa8c8'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:14.704205 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1379/2747] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:21:14 2022] DELETE /identity/v3/users/43cb895aebfd4cf2a079108fd29aa8c8 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:21:15.132367 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b88512-bbc3-4f0c-936a-a82e76f83734 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:15.134005 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b0b88512-bbc3-4f0c-936a-a82e76f83734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:21:15.150315 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b88512-bbc3-4f0c-936a-a82e76f83734 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:15.224822 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56a665-366e-4af5-8be6-766cebee572e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:15.225687 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56a665-366e-4af5-8be6-766cebee572e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:15.225977 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56a665-366e-4af5-8be6-766cebee572e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:15.226255 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56a665-366e-4af5-8be6-766cebee572e admin admin] PATH_INFO: `/v3/projects/00d59a2a28b64f00a3bc8d27ccd9ac0a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:15.227546 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a56a665-366e-4af5-8be6-766cebee572e admin admin] RBAC: Authorizing `identity:delete_project(project_id=00d59a2a28b64f00a3bc8d27ccd9ac0a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:15.229503 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a56a665-366e-4af5-8be6-766cebee572e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:15.378807 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4a56a665-366e-4af5-8be6-766cebee572e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00d59a2a28b64f00a3bc8d27ccd9ac0a', 'request_id': 'req-4a56a665-366e-4af5-8be6-766cebee572e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:15.383144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:21:15.450319 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4a56a665-366e-4af5-8be6-766cebee572e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00d59a2a28b64f00a3bc8d27ccd9ac0a', 'request_id': 'req-4a56a665-366e-4af5-8be6-766cebee572e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:15.459577 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4a56a665-366e-4af5-8be6-766cebee572e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00d59a2a28b64f00a3bc8d27ccd9ac0a', 'request_id': 'req-4a56a665-366e-4af5-8be6-766cebee572e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:15.471360 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4a56a665-366e-4af5-8be6-766cebee572e admin admin] The token cache is being invalidate because project 00d59a2a28b64f00a3bc8d27ccd9ac0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:21:15.471912 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4a56a665-366e-4af5-8be6-766cebee572e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:15.474325 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4a56a665-366e-4af5-8be6-766cebee572e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:15.476409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1369/2748] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:21:15 2022] DELETE /identity/v3/projects/00d59a2a28b64f00a3bc8d27ccd9ac0a => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:21:15.483769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81443a5d-80f9-42ef-b7d3-42571b1254dd None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:15.486342 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81443a5d-80f9-42ef-b7d3-42571b1254dd None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:15.486342 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81443a5d-80f9-42ef-b7d3-42571b1254dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:15.539701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-81443a5d-80f9-42ef-b7d3-42571b1254dd None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:21:15.541042 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-81443a5d-80f9-42ef-b7d3-42571b1254dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:21:15.550086 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:21:15.611246 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1380/2749] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:21:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:21:15.613234 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15268be-e405-4d45-ac50-7bd325536b8f None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:15.613555 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15268be-e405-4d45-ac50-7bd325536b8f None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:15.614521 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15268be-e405-4d45-ac50-7bd325536b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:15.636460 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-e15268be-e405-4d45-ac50-7bd325536b8f None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:21:15.637734 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e15268be-e405-4d45-ac50-7bd325536b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:21:15.647546 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1370/2750] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:21:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:21:15.651773 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81443a5d-80f9-42ef-b7d3-42571b1254dd None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:15.667919 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81443a5d-80f9-42ef-b7d3-42571b1254dd None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:15.674336 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de39078-d040-40f3-9df3-3527d7d79c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:15.675311 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de39078-d040-40f3-9df3-3527d7d79c46 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:15.675774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de39078-d040-40f3-9df3-3527d7d79c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:15.676278 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de39078-d040-40f3-9df3-3527d7d79c46 admin admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:15.676956 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de39078-d040-40f3-9df3-3527d7d79c46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:21:15.677266 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de39078-d040-40f3-9df3-3527d7d79c46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:15.678917 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de39078-d040-40f3-9df3-3527d7d79c46 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:15.691226 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1381/2751] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:21:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 05:21:15.696063 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15268be-e405-4d45-ac50-7bd325536b8f None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:15.707147 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15268be-e405-4d45-ac50-7bd325536b8f None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:15.712607 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b19b7-9c4e-4815-bec3-c269c90dc397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:15.713444 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b19b7-9c4e-4815-bec3-c269c90dc397 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:15.714082 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b19b7-9c4e-4815-bec3-c269c90dc397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:15.714082 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b19b7-9c4e-4815-bec3-c269c90dc397 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:15.714579 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7b19b7-9c4e-4815-bec3-c269c90dc397 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:15.716749 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7b19b7-9c4e-4815-bec3-c269c90dc397 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:15.858043 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1371/2752] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:21:15 2022] POST /identity/v3/projects => generated 385 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:21:15.867118 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de39078-d040-40f3-9df3-3527d7d79c46 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:15.876532 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de39078-d040-40f3-9df3-3527d7d79c46 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:15.946187 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59395efc-73a8-49d4-9857-66a3b8f2c9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:15.947260 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59395efc-73a8-49d4-9857-66a3b8f2c9aa admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:15.947549 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59395efc-73a8-49d4-9857-66a3b8f2c9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:15.947838 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59395efc-73a8-49d4-9857-66a3b8f2c9aa admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:15.948594 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59395efc-73a8-49d4-9857-66a3b8f2c9aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:15.950575 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59395efc-73a8-49d4-9857-66a3b8f2c9aa admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:16.047683 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1382/2753] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:21:15 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:21:16.050447 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b19b7-9c4e-4815-bec3-c269c90dc397 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:16.063172 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b19b7-9c4e-4815-bec3-c269c90dc397 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:16.068598 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a09ce-d9c3-4a59-8abd-04b7174cf701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:16.069227 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a09ce-d9c3-4a59-8abd-04b7174cf701 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:16.069519 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a09ce-d9c3-4a59-8abd-04b7174cf701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:16.069798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a09ce-d9c3-4a59-8abd-04b7174cf701 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:16.070502 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a09ce-d9c3-4a59-8abd-04b7174cf701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:16.072295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a09ce-d9c3-4a59-8abd-04b7174cf701 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:16.083943 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1372/2754] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:21:16.088290 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59395efc-73a8-49d4-9857-66a3b8f2c9aa admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:16.100847 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59395efc-73a8-49d4-9857-66a3b8f2c9aa admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:16.106941 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8838cff-ffa3-4f6b-96b6-c0eaf8c04a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:16.107488 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8838cff-ffa3-4f6b-96b6-c0eaf8c04a4c admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:16.107786 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8838cff-ffa3-4f6b-96b6-c0eaf8c04a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:16.108069 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8838cff-ffa3-4f6b-96b6-c0eaf8c04a4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:16.108777 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8838cff-ffa3-4f6b-96b6-c0eaf8c04a4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:16.110562 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8838cff-ffa3-4f6b-96b6-c0eaf8c04a4c admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:16.124856 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1383/2755] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:21:16.129913 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a09ce-d9c3-4a59-8abd-04b7174cf701 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:16.143804 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a09ce-d9c3-4a59-8abd-04b7174cf701 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:16.149057 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebfda25-bd3e-4f23-ab64-183d0f8bba45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:16.149810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebfda25-bd3e-4f23-ab64-183d0f8bba45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:16.150165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebfda25-bd3e-4f23-ab64-183d0f8bba45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:16.150554 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebfda25-bd3e-4f23-ab64-183d0f8bba45 admin admin] PATH_INFO: `/v3/projects/efbbe35b007f4fed88a44be892587c24/users/343e7f7f4d0e40ecb734d84cd7fef7a4/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:16.166341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebfda25-bd3e-4f23-ab64-183d0f8bba45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efbbe35b007f4fed88a44be892587c24, user_id=343e7f7f4d0e40ecb734d84cd7fef7a4, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:16.169273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebfda25-bd3e-4f23-ab64-183d0f8bba45 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:16.229638 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1373/2756] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:21:16 2022] PUT /identity/v3/projects/efbbe35b007f4fed88a44be892587c24/users/343e7f7f4d0e40ecb734d84cd7fef7a4/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:21:16.241078 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfdf08-b0a5-435a-b318-cde37e6a9684 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:16.241374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfdf08-b0a5-435a-b318-cde37e6a9684 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:16.241663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfdf08-b0a5-435a-b318-cde37e6a9684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:16.279409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-aecfdf08-b0a5-435a-b318-cde37e6a9684 None None] MFA Rules not processed for user `343e7f7f4d0e40ecb734d84cd7fef7a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:21:16.280749 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-aecfdf08-b0a5-435a-b318-cde37e6a9684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:21:16.356267 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1384/2757] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:21:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:21:16.368471 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebfda25-bd3e-4f23-ab64-183d0f8bba45 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:16.370195 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1ebfda25-bd3e-4f23-ab64-183d0f8bba45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:21:16.473344 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebfda25-bd3e-4f23-ab64-183d0f8bba45 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:16.477946 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac33428-1508-4109-ab61-ded1be6886c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:16.478660 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac33428-1508-4109-ab61-ded1be6886c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:16.479049 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac33428-1508-4109-ab61-ded1be6886c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:16.479415 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac33428-1508-4109-ab61-ded1be6886c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:16.493326 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac33428-1508-4109-ab61-ded1be6886c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:16.494971 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac33428-1508-4109-ab61-ded1be6886c9 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:16.505521 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1374/2758] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:21:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:21:19.655387 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecfdf08-b0a5-435a-b318-cde37e6a9684 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:19.666332 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:21:19.725326 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecfdf08-b0a5-435a-b318-cde37e6a9684 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:19.730636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d086b2-9c5b-491b-a666-8a98fa4c1484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:19.731422 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d086b2-9c5b-491b-a666-8a98fa4c1484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:19.731778 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d086b2-9c5b-491b-a666-8a98fa4c1484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:19.732125 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d086b2-9c5b-491b-a666-8a98fa4c1484 admin admin] PATH_INFO: `/v3/users/343e7f7f4d0e40ecb734d84cd7fef7a4` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:19.733380 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d086b2-9c5b-491b-a666-8a98fa4c1484 admin admin] RBAC: Authorizing `identity:delete_user(user_id=343e7f7f4d0e40ecb734d84cd7fef7a4)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:19.735523 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d086b2-9c5b-491b-a666-8a98fa4c1484 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:19.929690 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-98d086b2-9c5b-491b-a666-8a98fa4c1484 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '343e7f7f4d0e40ecb734d84cd7fef7a4'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:19.997987 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-98d086b2-9c5b-491b-a666-8a98fa4c1484 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '343e7f7f4d0e40ecb734d84cd7fef7a4'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:19.999021 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-98d086b2-9c5b-491b-a666-8a98fa4c1484 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '343e7f7f4d0e40ecb734d84cd7fef7a4'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:20.014894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-98d086b2-9c5b-491b-a666-8a98fa4c1484 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '343e7f7f4d0e40ecb734d84cd7fef7a4'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:20.015456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-98d086b2-9c5b-491b-a666-8a98fa4c1484 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '343e7f7f4d0e40ecb734d84cd7fef7a4'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:20.030052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1385/2759] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:21:19 2022] DELETE /identity/v3/users/343e7f7f4d0e40ecb734d84cd7fef7a4 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:21:20.034793 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b992e-6f2f-4f3c-a8e4-01166d5e3e3b None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:20.035855 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b992e-6f2f-4f3c-a8e4-01166d5e3e3b None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:20.035953 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b992e-6f2f-4f3c-a8e4-01166d5e3e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:20.056728 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-1e2b992e-6f2f-4f3c-a8e4-01166d5e3e3b None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:21:20.058206 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1e2b992e-6f2f-4f3c-a8e4-01166d5e3e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:21:20.130007 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1375/2760] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:21:20.137626 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d086b2-9c5b-491b-a666-8a98fa4c1484 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:20.139242 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-98d086b2-9c5b-491b-a666-8a98fa4c1484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:21:20.209748 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d086b2-9c5b-491b-a666-8a98fa4c1484 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:20.216370 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7dca0b-c8a9-412e-9966-342d1d9fa99d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:20.216370 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7dca0b-c8a9-412e-9966-342d1d9fa99d service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:20.216370 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7dca0b-c8a9-412e-9966-342d1d9fa99d service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:20.216370 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7dca0b-c8a9-412e-9966-342d1d9fa99d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:20.233801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7dca0b-c8a9-412e-9966-342d1d9fa99d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:20.233801 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7dca0b-c8a9-412e-9966-342d1d9fa99d service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:20.237701 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1386/2761] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:21:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:21:20.640056 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b992e-6f2f-4f3c-a8e4-01166d5e3e3b None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:20.642423 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1e2b992e-6f2f-4f3c-a8e4-01166d5e3e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:21:20.655369 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:21:20.661359 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b992e-6f2f-4f3c-a8e4-01166d5e3e3b None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:20.668758 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f74833-b108-49f4-9a34-2d1cc506d131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:20.669503 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f74833-b108-49f4-9a34-2d1cc506d131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:20.669804 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f74833-b108-49f4-9a34-2d1cc506d131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:20.670091 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f74833-b108-49f4-9a34-2d1cc506d131 admin admin] PATH_INFO: `/v3/projects/efbbe35b007f4fed88a44be892587c24` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:20.672102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f74833-b108-49f4-9a34-2d1cc506d131 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efbbe35b007f4fed88a44be892587c24)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:20.673711 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f74833-b108-49f4-9a34-2d1cc506d131 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:20.804119 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f5f74833-b108-49f4-9a34-2d1cc506d131 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efbbe35b007f4fed88a44be892587c24', 'request_id': 'req-f5f74833-b108-49f4-9a34-2d1cc506d131'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:20.867700 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f5f74833-b108-49f4-9a34-2d1cc506d131 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efbbe35b007f4fed88a44be892587c24', 'request_id': 'req-f5f74833-b108-49f4-9a34-2d1cc506d131'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:20.877202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f5f74833-b108-49f4-9a34-2d1cc506d131 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efbbe35b007f4fed88a44be892587c24', 'request_id': 'req-f5f74833-b108-49f4-9a34-2d1cc506d131'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:20.892402 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f5f74833-b108-49f4-9a34-2d1cc506d131 admin admin] The token cache is being invalidate because project efbbe35b007f4fed88a44be892587c24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:21:20.892835 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f5f74833-b108-49f4-9a34-2d1cc506d131 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:20.893617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f5f74833-b108-49f4-9a34-2d1cc506d131 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:20.896069 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1376/2762] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:21:20 2022] DELETE /identity/v3/projects/efbbe35b007f4fed88a44be892587c24 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:21:20.901695 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70b095-883d-4848-bb14-39e5304832ad None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:20.902685 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70b095-883d-4848-bb14-39e5304832ad None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:20.903176 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70b095-883d-4848-bb14-39e5304832ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:20.929580 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-2e70b095-883d-4848-bb14-39e5304832ad None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:21:20.931091 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-2e70b095-883d-4848-bb14-39e5304832ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:21:20.966758 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:21:21.011869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1387/2763] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:21:21.022480 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a699f-f3a3-4c44-afd4-3b6875a5b503 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:21.022774 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a699f-f3a3-4c44-afd4-3b6875a5b503 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:21.023099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a699f-f3a3-4c44-afd4-3b6875a5b503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:21.059262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-b55a699f-f3a3-4c44-afd4-3b6875a5b503 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:21:21.061278 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b55a699f-f3a3-4c44-afd4-3b6875a5b503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:21:21.080792 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1377/2764] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:21:21.092833 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70b095-883d-4848-bb14-39e5304832ad None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:21.103352 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70b095-883d-4848-bb14-39e5304832ad None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:21.108940 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c5e235-5f95-4e5e-b07e-32c77c360848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:21.109513 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5e235-5f95-4e5e-b07e-32c77c360848 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:21.109859 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5e235-5f95-4e5e-b07e-32c77c360848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:21.110132 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5e235-5f95-4e5e-b07e-32c77c360848 admin admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:21.110825 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c5e235-5f95-4e5e-b07e-32c77c360848 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:21:21.111190 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c5e235-5f95-4e5e-b07e-32c77c360848 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:21.112826 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c5e235-5f95-4e5e-b07e-32c77c360848 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:21.125602 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1388/2765] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:21:21.134416 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a699f-f3a3-4c44-afd4-3b6875a5b503 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:21.151064 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a699f-f3a3-4c44-afd4-3b6875a5b503 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:21.156098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38436960-b3be-43e2-be2d-f0e461eeef5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:21.156822 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38436960-b3be-43e2-be2d-f0e461eeef5e admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:21.172889 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38436960-b3be-43e2-be2d-f0e461eeef5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:21.172889 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38436960-b3be-43e2-be2d-f0e461eeef5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:21.172889 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38436960-b3be-43e2-be2d-f0e461eeef5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:21.172889 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38436960-b3be-43e2-be2d-f0e461eeef5e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:21.255595 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1378/2766] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:21:21 2022] POST /identity/v3/projects => generated 371 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:21:21.259648 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c5e235-5f95-4e5e-b07e-32c77c360848 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:21.270787 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c5e235-5f95-4e5e-b07e-32c77c360848 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:21.379724 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af3e9f-698c-4c4f-90fc-94fe27277554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:21.380489 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af3e9f-698c-4c4f-90fc-94fe27277554 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:21.380843 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af3e9f-698c-4c4f-90fc-94fe27277554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:21.381252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af3e9f-698c-4c4f-90fc-94fe27277554 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:21.381991 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af3e9f-698c-4c4f-90fc-94fe27277554 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:21.384085 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af3e9f-698c-4c4f-90fc-94fe27277554 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:21.465308 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1389/2767] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:21:21 2022] POST /identity/v3/users => generated 286 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:21:21.469154 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38436960-b3be-43e2-be2d-f0e461eeef5e admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:21.489542 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38436960-b3be-43e2-be2d-f0e461eeef5e admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:21.491900 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af3e9f-698c-4c4f-90fc-94fe27277554 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:21.493325 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-e8af3e9f-698c-4c4f-90fc-94fe27277554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:21:21.494810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a0e4d-3587-4150-bc23-7dbbbc793374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:21.495592 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a0e4d-3587-4150-bc23-7dbbbc793374 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:21.495965 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a0e4d-3587-4150-bc23-7dbbbc793374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:21.496320 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a0e4d-3587-4150-bc23-7dbbbc793374 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:21.497144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a0e4d-3587-4150-bc23-7dbbbc793374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:21.498810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a0e4d-3587-4150-bc23-7dbbbc793374 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:21.513186 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1379/2768] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:21:21.516953 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a0e4d-3587-4150-bc23-7dbbbc793374 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:21.529822 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a0e4d-3587-4150-bc23-7dbbbc793374 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:21.538299 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ccc679-23f8-452e-8f25-c3a4f16ec377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:21.540099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ccc679-23f8-452e-8f25-c3a4f16ec377 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:21.540469 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ccc679-23f8-452e-8f25-c3a4f16ec377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:21.543333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ccc679-23f8-452e-8f25-c3a4f16ec377 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:21.543333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ccc679-23f8-452e-8f25-c3a4f16ec377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:21.543559 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ccc679-23f8-452e-8f25-c3a4f16ec377 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:21.558588 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1380/2769] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:21:21.559532 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ccc679-23f8-452e-8f25-c3a4f16ec377 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:21.570759 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ccc679-23f8-452e-8f25-c3a4f16ec377 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:21.582909 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdde6e-ed3e-4567-a601-b57c0c67bb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:21.583647 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdde6e-ed3e-4567-a601-b57c0c67bb9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:21.583939 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdde6e-ed3e-4567-a601-b57c0c67bb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:21.584221 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdde6e-ed3e-4567-a601-b57c0c67bb9c admin admin] PATH_INFO: `/v3/projects/a43ed449ae0245d785bb962cb87921a5/users/0a73ee7e4a4f42b7a9db0bbc03d208f5/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:21.585203 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af3e9f-698c-4c4f-90fc-94fe27277554 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:21.602381 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06144d8c-7b58-40d2-8030-d35363b0dc09 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59846d88f4314c2382aa5cb8bae5053b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:21.603452 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06144d8c-7b58-40d2-8030-d35363b0dc09 service placement] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:21.603898 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06144d8c-7b58-40d2-8030-d35363b0dc09 service placement] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:21.604423 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06144d8c-7b58-40d2-8030-d35363b0dc09 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:21.607073 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-06144d8c-7b58-40d2-8030-d35363b0dc09 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:21:21.607239 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdde6e-ed3e-4567-a601-b57c0c67bb9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a43ed449ae0245d785bb962cb87921a5, user_id=0a73ee7e4a4f42b7a9db0bbc03d208f5, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:21.610104 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdde6e-ed3e-4567-a601-b57c0c67bb9c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:21.622016 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06144d8c-7b58-40d2-8030-d35363b0dc09 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:21.623784 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06144d8c-7b58-40d2-8030-d35363b0dc09 service placement] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:21.634899 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1390/2770] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:21:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:21:21.635869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06144d8c-7b58-40d2-8030-d35363b0dc09 service placement] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:21.637288 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-06144d8c-7b58-40d2-8030-d35363b0dc09 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:21:21.652333 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06144d8c-7b58-40d2-8030-d35363b0dc09 service placement] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:21.657048 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f8b23-bc50-479e-a107-c41c4a3e44fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:21.661517 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f8b23-bc50-479e-a107-c41c4a3e44fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:21.661517 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f8b23-bc50-479e-a107-c41c4a3e44fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:21.661517 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f8b23-bc50-479e-a107-c41c4a3e44fa admin admin] PATH_INFO: `/v3/users/f145cffb3c4d4965a35553b95b316f56` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:21.661517 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016f8b23-bc50-479e-a107-c41c4a3e44fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=f145cffb3c4d4965a35553b95b316f56)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:21.673566 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016f8b23-bc50-479e-a107-c41c4a3e44fa admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:21.686452 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1381/2771] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:21:21 2022] PUT /identity/v3/projects/a43ed449ae0245d785bb962cb87921a5/users/0a73ee7e4a4f42b7a9db0bbc03d208f5/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:21:21.697652 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13076235-5be2-45d7-8f99-5e223d4d1216 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:21.699096 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13076235-5be2-45d7-8f99-5e223d4d1216 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:21.699096 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13076235-5be2-45d7-8f99-5e223d4d1216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:21.729676 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-13076235-5be2-45d7-8f99-5e223d4d1216 None None] MFA Rules not processed for user `0a73ee7e4a4f42b7a9db0bbc03d208f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:21:21.730993 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-13076235-5be2-45d7-8f99-5e223d4d1216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:21:21.800402 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:21:21.805062 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1382/2772] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:21:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:21:21.816950 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13076235-5be2-45d7-8f99-5e223d4d1216 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:21.818476 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-13076235-5be2-45d7-8f99-5e223d4d1216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:21:21.845777 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:21:21.861032 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-016f8b23-bc50-479e-a107-c41c4a3e44fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f145cffb3c4d4965a35553b95b316f56'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:21.884881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13076235-5be2-45d7-8f99-5e223d4d1216 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:21.888610 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa8f77-7bd0-432f-8187-10abe9d25a1c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cc14b33895443b893fab487bfe1858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:21.889228 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa8f77-7bd0-432f-8187-10abe9d25a1c service glance] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:21.889515 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa8f77-7bd0-432f-8187-10abe9d25a1c service glance] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:21.889791 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa8f77-7bd0-432f-8187-10abe9d25a1c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:21.898656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa8f77-7bd0-432f-8187-10abe9d25a1c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:21.900124 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa8f77-7bd0-432f-8187-10abe9d25a1c service glance] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:21.909136 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1383/2773] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:21:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:21:21.922929 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-016f8b23-bc50-479e-a107-c41c4a3e44fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f145cffb3c4d4965a35553b95b316f56'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:21.923580 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-016f8b23-bc50-479e-a107-c41c4a3e44fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f145cffb3c4d4965a35553b95b316f56'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:21.946435 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-016f8b23-bc50-479e-a107-c41c4a3e44fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f145cffb3c4d4965a35553b95b316f56'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:21.946435 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-016f8b23-bc50-479e-a107-c41c4a3e44fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f145cffb3c4d4965a35553b95b316f56'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:21.949251 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1391/2774] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:21:21 2022] DELETE /identity/v3/users/f145cffb3c4d4965a35553b95b316f56 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:21:22.183783 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa8f77-7bd0-432f-8187-10abe9d25a1c service glance] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:22.259101 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa8f77-7bd0-432f-8187-10abe9d25a1c service glance] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:22.323217 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf5209-5a2e-47e6-bbbd-1e0010108060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:22.323925 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf5209-5a2e-47e6-bbbd-1e0010108060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:22.324142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf5209-5a2e-47e6-bbbd-1e0010108060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:22.324423 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf5209-5a2e-47e6-bbbd-1e0010108060 admin admin] PATH_INFO: `/v3/projects/6ff6f2c611f44256a7d85a8d99981d9d` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:22.325642 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bf5209-5a2e-47e6-bbbd-1e0010108060 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ff6f2c611f44256a7d85a8d99981d9d)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:22.327876 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bf5209-5a2e-47e6-bbbd-1e0010108060 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:22.439407 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-47bf5209-5a2e-47e6-bbbd-1e0010108060 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ff6f2c611f44256a7d85a8d99981d9d', 'request_id': 'req-47bf5209-5a2e-47e6-bbbd-1e0010108060'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:22.442470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:21:22.524545 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-47bf5209-5a2e-47e6-bbbd-1e0010108060 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ff6f2c611f44256a7d85a8d99981d9d', 'request_id': 'req-47bf5209-5a2e-47e6-bbbd-1e0010108060'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:22.531060 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-47bf5209-5a2e-47e6-bbbd-1e0010108060 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ff6f2c611f44256a7d85a8d99981d9d', 'request_id': 'req-47bf5209-5a2e-47e6-bbbd-1e0010108060'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:22.542151 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-47bf5209-5a2e-47e6-bbbd-1e0010108060 admin admin] The token cache is being invalidate because project 6ff6f2c611f44256a7d85a8d99981d9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:21:22.542445 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-47bf5209-5a2e-47e6-bbbd-1e0010108060 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:22.543122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-47bf5209-5a2e-47e6-bbbd-1e0010108060 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:22.544632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1384/2775] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:21:22 2022] DELETE /identity/v3/projects/6ff6f2c611f44256a7d85a8d99981d9d => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:21:22.548894 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f8b23-bc50-479e-a107-c41c4a3e44fa admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:22.553506 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-016f8b23-bc50-479e-a107-c41c4a3e44fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:21:22.563857 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f8b23-bc50-479e-a107-c41c4a3e44fa admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:22.622759 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583c1fc-dff3-40e2-baef-d67aafb085fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:22.623433 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583c1fc-dff3-40e2-baef-d67aafb085fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:22.623719 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583c1fc-dff3-40e2-baef-d67aafb085fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:22.623998 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583c1fc-dff3-40e2-baef-d67aafb085fd admin admin] PATH_INFO: `/v3/users/7a806f98beb5459085fd4cca1f28da96` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:22.625216 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c583c1fc-dff3-40e2-baef-d67aafb085fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a806f98beb5459085fd4cca1f28da96)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:22.627199 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c583c1fc-dff3-40e2-baef-d67aafb085fd admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:22.789555 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-c583c1fc-dff3-40e2-baef-d67aafb085fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a806f98beb5459085fd4cca1f28da96'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:22.992321 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-c583c1fc-dff3-40e2-baef-d67aafb085fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a806f98beb5459085fd4cca1f28da96'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:22.993089 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-c583c1fc-dff3-40e2-baef-d67aafb085fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a806f98beb5459085fd4cca1f28da96'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:23.006131 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-c583c1fc-dff3-40e2-baef-d67aafb085fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a806f98beb5459085fd4cca1f28da96'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:23.006792 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-c583c1fc-dff3-40e2-baef-d67aafb085fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a806f98beb5459085fd4cca1f28da96'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:23.019219 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1392/2776] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:21:22 2022] DELETE /identity/v3/users/7a806f98beb5459085fd4cca1f28da96 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:21:23.250644 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf5209-5a2e-47e6-bbbd-1e0010108060 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:23.252251 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-47bf5209-5a2e-47e6-bbbd-1e0010108060 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:21:23.267038 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf5209-5a2e-47e6-bbbd-1e0010108060 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:23.329865 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdbbf19-1409-42f9-9d82-5da48a29a773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:23.331728 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbbf19-1409-42f9-9d82-5da48a29a773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:23.331728 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbbf19-1409-42f9-9d82-5da48a29a773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:23.331728 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbbf19-1409-42f9-9d82-5da48a29a773 admin admin] PATH_INFO: `/v3/projects/2f9038a964ca4cd595ed4851dd9e6407` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:23.333267 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdbbf19-1409-42f9-9d82-5da48a29a773 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f9038a964ca4cd595ed4851dd9e6407)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:23.335625 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdbbf19-1409-42f9-9d82-5da48a29a773 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:23.496000 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4cdbbf19-1409-42f9-9d82-5da48a29a773 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f9038a964ca4cd595ed4851dd9e6407', 'request_id': 'req-4cdbbf19-1409-42f9-9d82-5da48a29a773'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:23.551213 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4cdbbf19-1409-42f9-9d82-5da48a29a773 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f9038a964ca4cd595ed4851dd9e6407', 'request_id': 'req-4cdbbf19-1409-42f9-9d82-5da48a29a773'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:23.557686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4cdbbf19-1409-42f9-9d82-5da48a29a773 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f9038a964ca4cd595ed4851dd9e6407', 'request_id': 'req-4cdbbf19-1409-42f9-9d82-5da48a29a773'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:23.569125 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4cdbbf19-1409-42f9-9d82-5da48a29a773 admin admin] The token cache is being invalidate because project 2f9038a964ca4cd595ed4851dd9e6407 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:21:23.569443 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4cdbbf19-1409-42f9-9d82-5da48a29a773 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:23.570098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4cdbbf19-1409-42f9-9d82-5da48a29a773 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:23.571664 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1385/2777] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:21:23 2022] DELETE /identity/v3/projects/2f9038a964ca4cd595ed4851dd9e6407 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:21:33.015563 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=70,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:21:33.566024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:DELETE=10,keystone:SELECT=57,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:21:51.125240 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583c1fc-dff3-40e2-baef-d67aafb085fd admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:51.126666 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-c583c1fc-dff3-40e2-baef-d67aafb085fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:21:51.140365 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583c1fc-dff3-40e2-baef-d67aafb085fd admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:51.196650 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ed208-14b2-48af-8531-50b897167b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:51.197301 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ed208-14b2-48af-8531-50b897167b82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:51.197589 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ed208-14b2-48af-8531-50b897167b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:51.197869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ed208-14b2-48af-8531-50b897167b82 admin admin] PATH_INFO: `/v3/users/0a73ee7e4a4f42b7a9db0bbc03d208f5` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:51.199102 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ed208-14b2-48af-8531-50b897167b82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a73ee7e4a4f42b7a9db0bbc03d208f5)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:51.201346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ed208-14b2-48af-8531-50b897167b82 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:51.376258 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a45ed208-14b2-48af-8531-50b897167b82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a73ee7e4a4f42b7a9db0bbc03d208f5'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:51.453842 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a45ed208-14b2-48af-8531-50b897167b82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a73ee7e4a4f42b7a9db0bbc03d208f5'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:51.454467 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a45ed208-14b2-48af-8531-50b897167b82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a73ee7e4a4f42b7a9db0bbc03d208f5'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:51.467325 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a45ed208-14b2-48af-8531-50b897167b82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a73ee7e4a4f42b7a9db0bbc03d208f5'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:51.467983 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a45ed208-14b2-48af-8531-50b897167b82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a73ee7e4a4f42b7a9db0bbc03d208f5'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:51.480769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1393/2778] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:21:51 2022] DELETE /identity/v3/users/0a73ee7e4a4f42b7a9db0bbc03d208f5 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:21:51.485421 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6e3cb-bb57-4e45-8ca5-cba289671d88 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:51.485872 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6e3cb-bb57-4e45-8ca5-cba289671d88 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:51.486019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6e3cb-bb57-4e45-8ca5-cba289671d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:51.506997 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-40a6e3cb-bb57-4e45-8ca5-cba289671d88 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:21:51.508239 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-40a6e3cb-bb57-4e45-8ca5-cba289671d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:21:51.571764 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1386/2779] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:21:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:21:51.581600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ed208-14b2-48af-8531-50b897167b82 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:51.583222 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-a45ed208-14b2-48af-8531-50b897167b82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:21:51.640022 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ed208-14b2-48af-8531-50b897167b82 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:51.643602 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a69369-73fe-43ef-906c-94abde785dcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:51.644222 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a69369-73fe-43ef-906c-94abde785dcc service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:51.644507 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a69369-73fe-43ef-906c-94abde785dcc service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:51.644786 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a69369-73fe-43ef-906c-94abde785dcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:51.654211 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a69369-73fe-43ef-906c-94abde785dcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:51.655697 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a69369-73fe-43ef-906c-94abde785dcc service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:51.666141 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1394/2780] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:21:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:21:52.036905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6e3cb-bb57-4e45-8ca5-cba289671d88 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:21:52.039259 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-40a6e3cb-bb57-4e45-8ca5-cba289671d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:21:52.054164 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6e3cb-bb57-4e45-8ca5-cba289671d88 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:21:52.058813 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532a470-4151-4968-9494-726a0568c571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:21:52.060022 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532a470-4151-4968-9494-726a0568c571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:21:52.060506 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532a470-4151-4968-9494-726a0568c571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:21:52.060795 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532a470-4151-4968-9494-726a0568c571 admin admin] PATH_INFO: `/v3/projects/a43ed449ae0245d785bb962cb87921a5` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:21:52.061978 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c532a470-4151-4968-9494-726a0568c571 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a43ed449ae0245d785bb962cb87921a5)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:21:52.063987 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c532a470-4151-4968-9494-726a0568c571 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:21:52.163710 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c532a470-4151-4968-9494-726a0568c571 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a43ed449ae0245d785bb962cb87921a5', 'request_id': 'req-c532a470-4151-4968-9494-726a0568c571'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:52.236719 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c532a470-4151-4968-9494-726a0568c571 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a43ed449ae0245d785bb962cb87921a5', 'request_id': 'req-c532a470-4151-4968-9494-726a0568c571'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:52.243051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c532a470-4151-4968-9494-726a0568c571 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a43ed449ae0245d785bb962cb87921a5', 'request_id': 'req-c532a470-4151-4968-9494-726a0568c571'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:52.254688 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c532a470-4151-4968-9494-726a0568c571 admin admin] The token cache is being invalidate because project a43ed449ae0245d785bb962cb87921a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:21:52.255089 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c532a470-4151-4968-9494-726a0568c571 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:52.255732 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c532a470-4151-4968-9494-726a0568c571 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:21:52.257760 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1387/2781] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:21:52 2022] DELETE /identity/v3/projects/a43ed449ae0245d785bb962cb87921a5 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:22:01.661056 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:22:02.251380 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:22:08.528496 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e0d8c-bcfd-4d24-8fc1-2246a13612eb None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:22:08.529044 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e0d8c-bcfd-4d24-8fc1-2246a13612eb None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:22:08.529044 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e0d8c-bcfd-4d24-8fc1-2246a13612eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:22:08.547938 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-475e0d8c-bcfd-4d24-8fc1-2246a13612eb None None] MFA Rules not processed for user `729a6d00e1664a6298088e52de9999f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:22:08.549199 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-475e0d8c-bcfd-4d24-8fc1-2246a13612eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:22:08.605434 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1395/2782] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:22:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:22:18.598235 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=25 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:22:57.556098 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532a470-4151-4968-9494-726a0568c571 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:22:57.557639 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c532a470-4151-4968-9494-726a0568c571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:22:57.615666 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532a470-4151-4968-9494-726a0568c571 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:22:57.619231 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4806ad-abce-4447-a6e9-b1e655575437 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:22:57.619894 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4806ad-abce-4447-a6e9-b1e655575437 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:22:57.620174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4806ad-abce-4447-a6e9-b1e655575437 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:22:57.620459 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4806ad-abce-4447-a6e9-b1e655575437 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:22:57.622215 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3a4806ad-abce-4447-a6e9-b1e655575437 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:22:57.695513 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4806ad-abce-4447-a6e9-b1e655575437 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:22:57.696969 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4806ad-abce-4447-a6e9-b1e655575437 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:22:57.708567 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1388/2783] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:22:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:22:59.336680 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e0d8c-bcfd-4d24-8fc1-2246a13612eb None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:22:59.338169 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-475e0d8c-bcfd-4d24-8fc1-2246a13612eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:22:59.393958 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e0d8c-bcfd-4d24-8fc1-2246a13612eb None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:22:59.398575 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6835a-edcd-4260-9db1-d2729f0c82d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:22:59.399216 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6835a-edcd-4260-9db1-d2729f0c82d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:22:59.399487 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6835a-edcd-4260-9db1-d2729f0c82d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:22:59.399760 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6835a-edcd-4260-9db1-d2729f0c82d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:22:59.401593 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-e9e6835a-edcd-4260-9db1-d2729f0c82d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:22:59.416267 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e6835a-edcd-4260-9db1-d2729f0c82d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:22:59.417775 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e6835a-edcd-4260-9db1-d2729f0c82d3 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:22:59.429934 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1396/2784] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:22:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:23:07.703475 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=54 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:23:09.424779 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=26 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:23:10.977104 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc2894-3d83-42cf-8b07-b2e6683dc5a1 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:23:10.977587 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc2894-3d83-42cf-8b07-b2e6683dc5a1 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:23:10.980958 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc2894-3d83-42cf-8b07-b2e6683dc5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:23:10.998597 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-0bdc2894-3d83-42cf-8b07-b2e6683dc5a1 None None] MFA Rules not processed for user `729a6d00e1664a6298088e52de9999f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:23:10.999867 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0bdc2894-3d83-42cf-8b07-b2e6683dc5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:23:11.007490 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1389/2785] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:23:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:23:16.106746 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6835a-edcd-4260-9db1-d2729f0c82d3 service neutron] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:23:16.121116 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6835a-edcd-4260-9db1-d2729f0c82d3 service neutron] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:23:16.129676 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77048d-30c5-4993-a4ce-960726ae9972 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:23:16.130473 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77048d-30c5-4993-a4ce-960726ae9972 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:23:16.131120 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77048d-30c5-4993-a4ce-960726ae9972 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:23:16.132234 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77048d-30c5-4993-a4ce-960726ae9972 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:23:16.134636 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-eb77048d-30c5-4993-a4ce-960726ae9972 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:23:16.148525 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77048d-30c5-4993-a4ce-960726ae9972 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:23:16.150461 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77048d-30c5-4993-a4ce-960726ae9972 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:23:16.164704 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1397/2786] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:23:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:23:18.181420 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa020844-09bf-4842-9b63-cf5d754b8153 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:23:18.181904 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa020844-09bf-4842-9b63-cf5d754b8153 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:23:18.182048 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa020844-09bf-4842-9b63-cf5d754b8153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:23:18.202656 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-fa020844-09bf-4842-9b63-cf5d754b8153 None None] MFA Rules not processed for user `ac3954ab39014ef3819111a6fe8142b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:23:18.204023 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-fa020844-09bf-4842-9b63-cf5d754b8153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:23:18.259887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1390/2787] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:23:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:23:26.156009 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=6 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:23:26.182435 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77048d-30c5-4993-a4ce-960726ae9972 service neutron] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:23:26.183909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-eb77048d-30c5-4993-a4ce-960726ae9972 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:23:26.207354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77048d-30c5-4993-a4ce-960726ae9972 service neutron] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:23:26.213616 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a63dd-c65a-4a2d-847c-d1d20e728ebf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:23:26.214233 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a63dd-c65a-4a2d-847c-d1d20e728ebf service nova] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:23:26.214520 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a63dd-c65a-4a2d-847c-d1d20e728ebf service nova] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:23:26.214802 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a63dd-c65a-4a2d-847c-d1d20e728ebf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:23:26.216611 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-3e6a63dd-c65a-4a2d-847c-d1d20e728ebf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:23:26.229440 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a63dd-c65a-4a2d-847c-d1d20e728ebf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:23:26.231178 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a63dd-c65a-4a2d-847c-d1d20e728ebf service nova] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:23:26.243401 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1398/2788] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:23:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:23:28.252870 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=28 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:23:36.238750 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=6 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:24:16.309651 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa020844-09bf-4842-9b63-cf5d754b8153 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:24:16.312663 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-fa020844-09bf-4842-9b63-cf5d754b8153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:24:16.332652 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa020844-09bf-4842-9b63-cf5d754b8153 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:24:16.338574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f71ff9-bb27-4678-8068-999d3e5da366 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:24:16.339241 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f71ff9-bb27-4678-8068-999d3e5da366 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:24:16.339524 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f71ff9-bb27-4678-8068-999d3e5da366 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:24:16.339784 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f71ff9-bb27-4678-8068-999d3e5da366 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:24:16.341589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a1f71ff9-bb27-4678-8068-999d3e5da366 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:24:16.357432 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f71ff9-bb27-4678-8068-999d3e5da366 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:24:16.359099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f71ff9-bb27-4678-8068-999d3e5da366 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:24:16.376644 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1391/2789] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:24:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:24:20.403876 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33743f5a-b43a-47b3-8290-c25e6ef0e6fc None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:24:20.404400 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33743f5a-b43a-47b3-8290-c25e6ef0e6fc None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:24:20.404400 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33743f5a-b43a-47b3-8290-c25e6ef0e6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:24:20.423655 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-33743f5a-b43a-47b3-8290-c25e6ef0e6fc None None] MFA Rules not processed for user `729a6d00e1664a6298088e52de9999f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:24:20.424855 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-33743f5a-b43a-47b3-8290-c25e6ef0e6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:24:20.431731 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1399/2790] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:24:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:24:26.370904 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=6 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:24:30.418869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:24:30.806190 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f71ff9-bb27-4678-8068-999d3e5da366 service nova] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:24:30.818063 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f71ff9-bb27-4678-8068-999d3e5da366 service nova] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:24:30.823796 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574c9abf-5d71-440c-a4a9-fd14cb7d50c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:24:30.824424 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c9abf-5d71-440c-a4a9-fd14cb7d50c5 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:24:30.824707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c9abf-5d71-440c-a4a9-fd14cb7d50c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:24:30.824984 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c9abf-5d71-440c-a4a9-fd14cb7d50c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:24:30.826797 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-574c9abf-5d71-440c-a4a9-fd14cb7d50c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:24:30.839335 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574c9abf-5d71-440c-a4a9-fd14cb7d50c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:24:30.843386 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574c9abf-5d71-440c-a4a9-fd14cb7d50c5 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:24:30.851994 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1392/2791] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:24:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:24:40.847580 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=6 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:25:25.933414 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56965cf-0894-4826-8621-e8e99118d1ee None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:25.933989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56965cf-0894-4826-8621-e8e99118d1ee None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:25.934195 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56965cf-0894-4826-8621-e8e99118d1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:25.957322 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-f56965cf-0894-4826-8621-e8e99118d1ee None None] MFA Rules not processed for user `ac3954ab39014ef3819111a6fe8142b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:25:25.958582 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-f56965cf-0894-4826-8621-e8e99118d1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:25.966541 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1400/2792] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:25:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:25:26.113299 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574c9abf-5d71-440c-a4a9-fd14cb7d50c5 service nova] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:26.122653 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574c9abf-5d71-440c-a4a9-fd14cb7d50c5 service nova] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:26.126291 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dbd348-f780-48e1-ab1a-5596dbe43268 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:26.126808 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbd348-f780-48e1-ab1a-5596dbe43268 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:26.127114 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbd348-f780-48e1-ab1a-5596dbe43268 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:26.127391 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbd348-f780-48e1-ab1a-5596dbe43268 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:26.129394 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-07dbd348-f780-48e1-ab1a-5596dbe43268 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:26.140529 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dbd348-f780-48e1-ab1a-5596dbe43268 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:26.142089 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dbd348-f780-48e1-ab1a-5596dbe43268 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:26.152004 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1393/2793] 10.222.0.40 () {60 vars in 1396 bytes} [Wed Aug 3 05:25:26 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3254 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:25:27.063913 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56965cf-0894-4826-8621-e8e99118d1ee None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:27.065481 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-f56965cf-0894-4826-8621-e8e99118d1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:27.140715 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56965cf-0894-4826-8621-e8e99118d1ee None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:27.146225 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b057d67-a6e6-4815-b494-9f6ca8b1ab58 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59846d88f4314c2382aa5cb8bae5053b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:27.147508 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b057d67-a6e6-4815-b494-9f6ca8b1ab58 service placement] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:27.147791 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b057d67-a6e6-4815-b494-9f6ca8b1ab58 service placement] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:27.148066 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b057d67-a6e6-4815-b494-9f6ca8b1ab58 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:27.149814 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-1b057d67-a6e6-4815-b494-9f6ca8b1ab58 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:27.160878 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b057d67-a6e6-4815-b494-9f6ca8b1ab58 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:27.162482 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b057d67-a6e6-4815-b494-9f6ca8b1ab58 service placement] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:27.172154 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1401/2794] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:25:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:25:36.147012 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=6 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:25:36.320172 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dbd348-f780-48e1-ab1a-5596dbe43268 service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:36.321805 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-07dbd348-f780-48e1-ab1a-5596dbe43268 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:36.389511 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dbd348-f780-48e1-ab1a-5596dbe43268 service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:36.394495 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5312146-d726-4bfb-bfb7-da33d1db5d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:36.395182 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5312146-d726-4bfb-bfb7-da33d1db5d0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:36.395461 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5312146-d726-4bfb-bfb7-da33d1db5d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:36.395732 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5312146-d726-4bfb-bfb7-da33d1db5d0f admin admin] PATH_INFO: `/v3/users/729a6d00e1664a6298088e52de9999f5` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:36.397175 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5312146-d726-4bfb-bfb7-da33d1db5d0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=729a6d00e1664a6298088e52de9999f5)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:36.399132 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5312146-d726-4bfb-bfb7-da33d1db5d0f admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:36.590467 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e5312146-d726-4bfb-bfb7-da33d1db5d0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '729a6d00e1664a6298088e52de9999f5'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:36.661317 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e5312146-d726-4bfb-bfb7-da33d1db5d0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '729a6d00e1664a6298088e52de9999f5'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:36.662658 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e5312146-d726-4bfb-bfb7-da33d1db5d0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '729a6d00e1664a6298088e52de9999f5'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:36.677495 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e5312146-d726-4bfb-bfb7-da33d1db5d0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '729a6d00e1664a6298088e52de9999f5'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:36.678225 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e5312146-d726-4bfb-bfb7-da33d1db5d0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '729a6d00e1664a6298088e52de9999f5'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:36.691027 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1394/2795] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:25:36 2022] DELETE /identity/v3/users/729a6d00e1664a6298088e52de9999f5 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:25:36.919645 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b057d67-a6e6-4815-b494-9f6ca8b1ab58 service placement] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:36.931995 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b057d67-a6e6-4815-b494-9f6ca8b1ab58 service placement] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:36.998862 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a4629-843d-4fd9-8c8e-b7f2b29d1b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:36.999518 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a4629-843d-4fd9-8c8e-b7f2b29d1b1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:36.999802 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a4629-843d-4fd9-8c8e-b7f2b29d1b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:37.000098 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a4629-843d-4fd9-8c8e-b7f2b29d1b1c admin admin] PATH_INFO: `/v3/projects/041a553912c64de981d9f87b551c190f` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:37.001254 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a4629-843d-4fd9-8c8e-b7f2b29d1b1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=041a553912c64de981d9f87b551c190f)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:37.003358 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a4629-843d-4fd9-8c8e-b7f2b29d1b1c admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:37.102566 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a22a4629-843d-4fd9-8c8e-b7f2b29d1b1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '041a553912c64de981d9f87b551c190f', 'request_id': 'req-a22a4629-843d-4fd9-8c8e-b7f2b29d1b1c'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:37.109086 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a22a4629-843d-4fd9-8c8e-b7f2b29d1b1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '041a553912c64de981d9f87b551c190f', 'request_id': 'req-a22a4629-843d-4fd9-8c8e-b7f2b29d1b1c'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:37.169828 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a22a4629-843d-4fd9-8c8e-b7f2b29d1b1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '041a553912c64de981d9f87b551c190f', 'request_id': 'req-a22a4629-843d-4fd9-8c8e-b7f2b29d1b1c'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:37.180788 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a22a4629-843d-4fd9-8c8e-b7f2b29d1b1c admin admin] The token cache is being invalidate because project 041a553912c64de981d9f87b551c190f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:25:37.181880 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a22a4629-843d-4fd9-8c8e-b7f2b29d1b1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:37.181880 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-a22a4629-843d-4fd9-8c8e-b7f2b29d1b1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:37.183409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1402/2796] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:25:36 2022] DELETE /identity/v3/projects/041a553912c64de981d9f87b551c190f => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:25:37.188466 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57feba5-9b56-4898-8e81-c99317885eef None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:37.188966 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57feba5-9b56-4898-8e81-c99317885eef None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:37.189071 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57feba5-9b56-4898-8e81-c99317885eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:37.209828 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-f57feba5-9b56-4898-8e81-c99317885eef None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:25:37.211082 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f57feba5-9b56-4898-8e81-c99317885eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:37.270765 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1395/2797] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:25:37.278755 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ecb6e-59a9-4bf9-bbb4-0435e3017f19 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:37.279081 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ecb6e-59a9-4bf9-bbb4-0435e3017f19 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:37.279368 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ecb6e-59a9-4bf9-bbb4-0435e3017f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:37.300214 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-c16ecb6e-59a9-4bf9-bbb4-0435e3017f19 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:25:37.302297 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-c16ecb6e-59a9-4bf9-bbb4-0435e3017f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:37.313391 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1403/2798] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:25:37.318728 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57feba5-9b56-4898-8e81-c99317885eef None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:37.328580 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57feba5-9b56-4898-8e81-c99317885eef None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:37.333128 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5267d82-ea59-47a6-b1cf-e9037ca3a8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:37.333733 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5267d82-ea59-47a6-b1cf-e9037ca3a8e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:37.334014 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5267d82-ea59-47a6-b1cf-e9037ca3a8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:37.334288 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5267d82-ea59-47a6-b1cf-e9037ca3a8e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:37.335002 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5267d82-ea59-47a6-b1cf-e9037ca3a8e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:25:37.335306 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5267d82-ea59-47a6-b1cf-e9037ca3a8e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:37.336896 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5267d82-ea59-47a6-b1cf-e9037ca3a8e9 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:37.348247 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1396/2799] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:25:37.352518 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ecb6e-59a9-4bf9-bbb4-0435e3017f19 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:37.361798 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ecb6e-59a9-4bf9-bbb4-0435e3017f19 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:37.367040 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f44f2-596d-46e4-a811-7f0718f75021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:37.367643 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f44f2-596d-46e4-a811-7f0718f75021 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:37.367928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f44f2-596d-46e4-a811-7f0718f75021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:37.368209 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f44f2-596d-46e4-a811-7f0718f75021 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:37.368869 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f44f2-596d-46e4-a811-7f0718f75021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:37.370790 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f44f2-596d-46e4-a811-7f0718f75021 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:37.375047 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:25:37.467735 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1404/2800] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:25:37 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:25:37.472019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5267d82-ea59-47a6-b1cf-e9037ca3a8e9 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:37.481995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5267d82-ea59-47a6-b1cf-e9037ca3a8e9 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:37.486221 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:25:37.540648 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0790ae6-2268-45f0-bf43-94bbf75c30e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:37.541178 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0790ae6-2268-45f0-bf43-94bbf75c30e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:37.541458 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0790ae6-2268-45f0-bf43-94bbf75c30e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:37.541734 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0790ae6-2268-45f0-bf43-94bbf75c30e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:37.542390 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0790ae6-2268-45f0-bf43-94bbf75c30e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:37.544463 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0790ae6-2268-45f0-bf43-94bbf75c30e0 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:37.646154 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1397/2801] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:25:37 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:25:37.650386 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f44f2-596d-46e4-a811-7f0718f75021 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:37.662929 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f44f2-596d-46e4-a811-7f0718f75021 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:37.667870 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6de208-2c4b-4bda-a0de-46380ec12a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:37.668529 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6de208-2c4b-4bda-a0de-46380ec12a19 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:37.668813 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6de208-2c4b-4bda-a0de-46380ec12a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:37.669090 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6de208-2c4b-4bda-a0de-46380ec12a19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:37.669786 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6de208-2c4b-4bda-a0de-46380ec12a19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:37.671494 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6de208-2c4b-4bda-a0de-46380ec12a19 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:37.683437 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1405/2802] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:25:37.687575 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0790ae6-2268-45f0-bf43-94bbf75c30e0 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:37.698209 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0790ae6-2268-45f0-bf43-94bbf75c30e0 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:37.705745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f640a39-3276-4023-81a2-58c39078fc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:37.705745 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f640a39-3276-4023-81a2-58c39078fc6a admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:37.705979 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f640a39-3276-4023-81a2-58c39078fc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:37.706099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f640a39-3276-4023-81a2-58c39078fc6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:37.706967 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f640a39-3276-4023-81a2-58c39078fc6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:37.708576 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f640a39-3276-4023-81a2-58c39078fc6a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:37.720405 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1398/2803] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:25:37.725075 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6de208-2c4b-4bda-a0de-46380ec12a19 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:37.735390 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6de208-2c4b-4bda-a0de-46380ec12a19 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:37.740468 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fcd2c-40f1-4b8d-aba7-df6687059246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:37.741148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fcd2c-40f1-4b8d-aba7-df6687059246 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:37.741439 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fcd2c-40f1-4b8d-aba7-df6687059246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:37.741722 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fcd2c-40f1-4b8d-aba7-df6687059246 admin admin] PATH_INFO: `/v3/projects/4af419fadcee40c98ef886109624a631/users/4de01cf3881741068c9ca8061696ed2a/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:37.757903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4fcd2c-40f1-4b8d-aba7-df6687059246 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4af419fadcee40c98ef886109624a631, user_id=4de01cf3881741068c9ca8061696ed2a, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:37.760826 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4fcd2c-40f1-4b8d-aba7-df6687059246 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:37.873544 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1406/2804] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:25:37 2022] PUT /identity/v3/projects/4af419fadcee40c98ef886109624a631/users/4de01cf3881741068c9ca8061696ed2a/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:25:37.884028 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea6422-c7ca-47c7-87d4-8871ad377e43 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:37.884317 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea6422-c7ca-47c7-87d4-8871ad377e43 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:37.884598 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea6422-c7ca-47c7-87d4-8871ad377e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:37.913581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-9aea6422-c7ca-47c7-87d4-8871ad377e43 None None] MFA Rules not processed for user `4de01cf3881741068c9ca8061696ed2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:25:37.914806 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9aea6422-c7ca-47c7-87d4-8871ad377e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:37.979581 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1399/2805] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:25:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:25:37.991793 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fcd2c-40f1-4b8d-aba7-df6687059246 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:37.993589 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-2f4fcd2c-40f1-4b8d-aba7-df6687059246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:38.089062 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fcd2c-40f1-4b8d-aba7-df6687059246 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:38.094651 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f433d2-63e3-4add-8d4e-d9b08ea01574 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:38.095789 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f433d2-63e3-4add-8d4e-d9b08ea01574 service nova] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:38.095789 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f433d2-63e3-4add-8d4e-d9b08ea01574 service nova] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:38.095981 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f433d2-63e3-4add-8d4e-d9b08ea01574 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:38.104663 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f433d2-63e3-4add-8d4e-d9b08ea01574 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:38.106310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f433d2-63e3-4add-8d4e-d9b08ea01574 service nova] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:38.115263 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1407/2806] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:25:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:25:46.111153 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea6422-c7ca-47c7-87d4-8871ad377e43 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:46.182495 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea6422-c7ca-47c7-87d4-8871ad377e43 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:46.186970 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72398745-83c8-4e25-81ca-319a4baa550d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:46.187616 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72398745-83c8-4e25-81ca-319a4baa550d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:46.187899 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72398745-83c8-4e25-81ca-319a4baa550d admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:46.188180 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72398745-83c8-4e25-81ca-319a4baa550d admin admin] PATH_INFO: `/v3/users/4de01cf3881741068c9ca8061696ed2a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:46.189358 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72398745-83c8-4e25-81ca-319a4baa550d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4de01cf3881741068c9ca8061696ed2a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:46.191586 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72398745-83c8-4e25-81ca-319a4baa550d admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:46.209782 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:25:46.331615 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-72398745-83c8-4e25-81ca-319a4baa550d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4de01cf3881741068c9ca8061696ed2a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:46.368892 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-72398745-83c8-4e25-81ca-319a4baa550d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4de01cf3881741068c9ca8061696ed2a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:46.370168 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-72398745-83c8-4e25-81ca-319a4baa550d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4de01cf3881741068c9ca8061696ed2a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:46.384490 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-72398745-83c8-4e25-81ca-319a4baa550d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4de01cf3881741068c9ca8061696ed2a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:46.385375 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-72398745-83c8-4e25-81ca-319a4baa550d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4de01cf3881741068c9ca8061696ed2a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:46.397898 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1400/2807] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:25:46 2022] DELETE /identity/v3/users/4de01cf3881741068c9ca8061696ed2a => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:25:46.402539 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058f82c-5dc0-43f7-b16b-308455c7442f None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:46.403085 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058f82c-5dc0-43f7-b16b-308455c7442f None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:46.403188 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058f82c-5dc0-43f7-b16b-308455c7442f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:46.422766 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-8058f82c-5dc0-43f7-b16b-308455c7442f None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:25:46.424025 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-8058f82c-5dc0-43f7-b16b-308455c7442f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:46.484797 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1408/2808] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:25:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:25:46.495173 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72398745-83c8-4e25-81ca-319a4baa550d admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:46.496684 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-72398745-83c8-4e25-81ca-319a4baa550d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:46.554487 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72398745-83c8-4e25-81ca-319a4baa550d admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:46.558364 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b468469-6c6b-424c-a314-9de2dfdf904b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:46.559024 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b468469-6c6b-424c-a314-9de2dfdf904b service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:46.559314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b468469-6c6b-424c-a314-9de2dfdf904b service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:46.559593 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b468469-6c6b-424c-a314-9de2dfdf904b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:46.570960 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b468469-6c6b-424c-a314-9de2dfdf904b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:46.573282 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b468469-6c6b-424c-a314-9de2dfdf904b service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:46.583116 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1401/2809] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:25:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:25:46.956642 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058f82c-5dc0-43f7-b16b-308455c7442f None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:46.958295 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-8058f82c-5dc0-43f7-b16b-308455c7442f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:46.974816 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058f82c-5dc0-43f7-b16b-308455c7442f None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:46.980002 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149cc7f-43db-46f7-b712-d92300c7168b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:46.980654 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149cc7f-43db-46f7-b712-d92300c7168b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:46.980943 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149cc7f-43db-46f7-b712-d92300c7168b admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:46.981215 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149cc7f-43db-46f7-b712-d92300c7168b admin admin] PATH_INFO: `/v3/projects/4af419fadcee40c98ef886109624a631` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:46.982430 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149cc7f-43db-46f7-b712-d92300c7168b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4af419fadcee40c98ef886109624a631)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:46.984420 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149cc7f-43db-46f7-b712-d92300c7168b admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:47.011974 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:25:47.102702 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-7149cc7f-43db-46f7-b712-d92300c7168b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4af419fadcee40c98ef886109624a631', 'request_id': 'req-7149cc7f-43db-46f7-b712-d92300c7168b'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:47.109311 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-7149cc7f-43db-46f7-b712-d92300c7168b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4af419fadcee40c98ef886109624a631', 'request_id': 'req-7149cc7f-43db-46f7-b712-d92300c7168b'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:47.174815 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-7149cc7f-43db-46f7-b712-d92300c7168b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4af419fadcee40c98ef886109624a631', 'request_id': 'req-7149cc7f-43db-46f7-b712-d92300c7168b'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:47.186725 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-7149cc7f-43db-46f7-b712-d92300c7168b admin admin] The token cache is being invalidate because project 4af419fadcee40c98ef886109624a631 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:25:47.192183 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-7149cc7f-43db-46f7-b712-d92300c7168b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:47.192183 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-7149cc7f-43db-46f7-b712-d92300c7168b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:47.192183 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1409/2810] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:25:46 2022] DELETE /identity/v3/projects/4af419fadcee40c98ef886109624a631 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:25:47.194530 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0fb2d-022b-4176-8807-4216c4caa929 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:47.194998 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0fb2d-022b-4176-8807-4216c4caa929 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:47.195142 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0fb2d-022b-4176-8807-4216c4caa929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:47.215412 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-2bb0fb2d-022b-4176-8807-4216c4caa929 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:25:47.216624 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2bb0fb2d-022b-4176-8807-4216c4caa929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:47.278961 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1402/2811] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:25:47.288083 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633be33d-7f26-4d47-a974-e92fd8803d72 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:47.288456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633be33d-7f26-4d47-a974-e92fd8803d72 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:47.288747 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633be33d-7f26-4d47-a974-e92fd8803d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:47.309854 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-633be33d-7f26-4d47-a974-e92fd8803d72 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:25:47.311106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-633be33d-7f26-4d47-a974-e92fd8803d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:47.319967 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1410/2812] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:25:47.324201 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb0fb2d-022b-4176-8807-4216c4caa929 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:47.334704 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb0fb2d-022b-4176-8807-4216c4caa929 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:47.339297 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16edd65c-32b4-43c8-a4e3-cb648b7435b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:47.339908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16edd65c-32b4-43c8-a4e3-cb648b7435b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:47.340189 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16edd65c-32b4-43c8-a4e3-cb648b7435b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:47.340465 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16edd65c-32b4-43c8-a4e3-cb648b7435b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:47.341156 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16edd65c-32b4-43c8-a4e3-cb648b7435b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:25:47.341461 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16edd65c-32b4-43c8-a4e3-cb648b7435b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:47.343140 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16edd65c-32b4-43c8-a4e3-cb648b7435b4 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:47.355888 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1403/2813] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:25:47.360314 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633be33d-7f26-4d47-a974-e92fd8803d72 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:47.369683 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633be33d-7f26-4d47-a974-e92fd8803d72 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:47.374351 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d35e7-3acf-4425-9fc1-4fda0558ad6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:47.375177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d35e7-3acf-4425-9fc1-4fda0558ad6d admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:47.375457 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d35e7-3acf-4425-9fc1-4fda0558ad6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:47.375796 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d35e7-3acf-4425-9fc1-4fda0558ad6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:47.376479 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d35e7-3acf-4425-9fc1-4fda0558ad6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:47.378393 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d35e7-3acf-4425-9fc1-4fda0558ad6d admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:47.447030 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1411/2814] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:25:47 2022] POST /identity/v3/projects => generated 373 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:25:47.453955 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16edd65c-32b4-43c8-a4e3-cb648b7435b4 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:47.465132 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16edd65c-32b4-43c8-a4e3-cb648b7435b4 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:47.486821 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:25:47.526763 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9a302-7d13-4a8a-9fc0-b24fb7e68a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:47.527444 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9a302-7d13-4a8a-9fc0-b24fb7e68a7e admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:47.527729 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9a302-7d13-4a8a-9fc0-b24fb7e68a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:47.528003 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9a302-7d13-4a8a-9fc0-b24fb7e68a7e admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:47.528673 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9a302-7d13-4a8a-9fc0-b24fb7e68a7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:47.530593 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9a302-7d13-4a8a-9fc0-b24fb7e68a7e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:47.615876 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1404/2815] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:25:47 2022] POST /identity/v3/users => generated 287 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:25:47.620164 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d35e7-3acf-4425-9fc1-4fda0558ad6d admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:47.631215 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d35e7-3acf-4425-9fc1-4fda0558ad6d admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:47.636274 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533de059-c7d8-4c74-a7b6-147f1651e911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:47.638369 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533de059-c7d8-4c74-a7b6-147f1651e911 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:47.638369 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533de059-c7d8-4c74-a7b6-147f1651e911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:47.638369 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533de059-c7d8-4c74-a7b6-147f1651e911 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:47.638369 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533de059-c7d8-4c74-a7b6-147f1651e911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:47.643577 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533de059-c7d8-4c74-a7b6-147f1651e911 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:47.654049 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1412/2816] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:25:47.658097 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9a302-7d13-4a8a-9fc0-b24fb7e68a7e admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:47.669603 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9a302-7d13-4a8a-9fc0-b24fb7e68a7e admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:47.674590 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae9164-3de3-4ff5-a2ea-392259d67c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:47.675298 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae9164-3de3-4ff5-a2ea-392259d67c28 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:47.677485 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae9164-3de3-4ff5-a2ea-392259d67c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:47.677485 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae9164-3de3-4ff5-a2ea-392259d67c28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:47.677485 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae9164-3de3-4ff5-a2ea-392259d67c28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:47.678393 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae9164-3de3-4ff5-a2ea-392259d67c28 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:47.689340 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1405/2817] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 05:25:47.694174 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533de059-c7d8-4c74-a7b6-147f1651e911 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:47.705265 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533de059-c7d8-4c74-a7b6-147f1651e911 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:47.714678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d29c2f-812e-4036-91e9-042ca1b81a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:47.714678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d29c2f-812e-4036-91e9-042ca1b81a96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:47.714678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d29c2f-812e-4036-91e9-042ca1b81a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:47.714678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d29c2f-812e-4036-91e9-042ca1b81a96 admin admin] PATH_INFO: `/v3/projects/1368716392674f62bd5b38d607455955/users/54c34855b8cc48fb9fc5febf917330a1/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:47.727490 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d29c2f-812e-4036-91e9-042ca1b81a96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1368716392674f62bd5b38d607455955, user_id=54c34855b8cc48fb9fc5febf917330a1, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:47.730418 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d29c2f-812e-4036-91e9-042ca1b81a96 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:47.804763 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1413/2818] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:25:47 2022] PUT /identity/v3/projects/1368716392674f62bd5b38d607455955/users/54c34855b8cc48fb9fc5febf917330a1/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:25:47.815002 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49021d85-c946-4f79-8878-5c086ed6813a None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:47.815587 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49021d85-c946-4f79-8878-5c086ed6813a None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:47.815988 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49021d85-c946-4f79-8878-5c086ed6813a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:47.844902 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-49021d85-c946-4f79-8878-5c086ed6813a None None] MFA Rules not processed for user `54c34855b8cc48fb9fc5febf917330a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:25:47.846135 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-49021d85-c946-4f79-8878-5c086ed6813a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:47.913585 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1406/2819] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:25:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:25:47.924891 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d29c2f-812e-4036-91e9-042ca1b81a96 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:47.927279 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-67d29c2f-812e-4036-91e9-042ca1b81a96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:48.006363 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d29c2f-812e-4036-91e9-042ca1b81a96 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:48.012156 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaac5b2-573f-4278-8736-68cd42e9bb9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3fa4a6c16274c18849869b55b33ac62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:48.014756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaac5b2-573f-4278-8736-68cd42e9bb9d service nova] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:48.014756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaac5b2-573f-4278-8736-68cd42e9bb9d service nova] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:48.014756 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaac5b2-573f-4278-8736-68cd42e9bb9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:48.022104 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaac5b2-573f-4278-8736-68cd42e9bb9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:48.023744 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaac5b2-573f-4278-8736-68cd42e9bb9d service nova] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:48.032950 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1414/2820] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:25:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:25:48.265228 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49021d85-c946-4f79-8878-5c086ed6813a None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:48.332297 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49021d85-c946-4f79-8878-5c086ed6813a None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:48.337966 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0686f7-0e96-4b3f-a9b5-d83b483204a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:48.338607 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0686f7-0e96-4b3f-a9b5-d83b483204a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:48.338909 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0686f7-0e96-4b3f-a9b5-d83b483204a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:48.339183 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0686f7-0e96-4b3f-a9b5-d83b483204a6 admin admin] PATH_INFO: `/v3/users/54c34855b8cc48fb9fc5febf917330a1` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:48.340354 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0686f7-0e96-4b3f-a9b5-d83b483204a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54c34855b8cc48fb9fc5febf917330a1)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:48.342287 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0686f7-0e96-4b3f-a9b5-d83b483204a6 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:48.524660 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:25:48.533937 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ed0686f7-0e96-4b3f-a9b5-d83b483204a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54c34855b8cc48fb9fc5febf917330a1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:48.597918 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ed0686f7-0e96-4b3f-a9b5-d83b483204a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54c34855b8cc48fb9fc5febf917330a1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:48.598601 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ed0686f7-0e96-4b3f-a9b5-d83b483204a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54c34855b8cc48fb9fc5febf917330a1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:48.611986 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ed0686f7-0e96-4b3f-a9b5-d83b483204a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54c34855b8cc48fb9fc5febf917330a1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:48.612690 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ed0686f7-0e96-4b3f-a9b5-d83b483204a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54c34855b8cc48fb9fc5febf917330a1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:48.625929 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1407/2821] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:25:48 2022] DELETE /identity/v3/users/54c34855b8cc48fb9fc5febf917330a1 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:25:48.630376 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9dd5c4-b5a1-4515-a8a3-c3e14c4e0575 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:48.630959 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9dd5c4-b5a1-4515-a8a3-c3e14c4e0575 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:48.631190 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9dd5c4-b5a1-4515-a8a3-c3e14c4e0575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:48.651209 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-ca9dd5c4-b5a1-4515-a8a3-c3e14c4e0575 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:25:48.652419 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-ca9dd5c4-b5a1-4515-a8a3-c3e14c4e0575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:48.680294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:25:48.714896 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1415/2822] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:25:48.725213 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0686f7-0e96-4b3f-a9b5-d83b483204a6 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:48.728128 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ed0686f7-0e96-4b3f-a9b5-d83b483204a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:48.788936 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0686f7-0e96-4b3f-a9b5-d83b483204a6 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:48.793114 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a688f0-cf20-41de-89f9-81058b6cb57b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:48.793733 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a688f0-cf20-41de-89f9-81058b6cb57b service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:48.794016 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a688f0-cf20-41de-89f9-81058b6cb57b service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:48.794289 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a688f0-cf20-41de-89f9-81058b6cb57b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:48.804158 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a688f0-cf20-41de-89f9-81058b6cb57b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:48.805865 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a688f0-cf20-41de-89f9-81058b6cb57b service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:48.816099 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1408/2823] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:25:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:25:49.546256 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9dd5c4-b5a1-4515-a8a3-c3e14c4e0575 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:49.549970 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-ca9dd5c4-b5a1-4515-a8a3-c3e14c4e0575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:49.581136 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9dd5c4-b5a1-4515-a8a3-c3e14c4e0575 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:49.589247 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52deb477-9119-47d2-a07d-0241fcda99a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:49.589800 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52deb477-9119-47d2-a07d-0241fcda99a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:49.590818 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52deb477-9119-47d2-a07d-0241fcda99a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:49.591570 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52deb477-9119-47d2-a07d-0241fcda99a9 admin admin] PATH_INFO: `/v3/projects/1368716392674f62bd5b38d607455955` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:49.594240 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52deb477-9119-47d2-a07d-0241fcda99a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1368716392674f62bd5b38d607455955)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:49.596254 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52deb477-9119-47d2-a07d-0241fcda99a9 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:49.710059 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-52deb477-9119-47d2-a07d-0241fcda99a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1368716392674f62bd5b38d607455955', 'request_id': 'req-52deb477-9119-47d2-a07d-0241fcda99a9'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:49.718629 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-52deb477-9119-47d2-a07d-0241fcda99a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1368716392674f62bd5b38d607455955', 'request_id': 'req-52deb477-9119-47d2-a07d-0241fcda99a9'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:49.797670 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-52deb477-9119-47d2-a07d-0241fcda99a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1368716392674f62bd5b38d607455955', 'request_id': 'req-52deb477-9119-47d2-a07d-0241fcda99a9'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:49.812081 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-52deb477-9119-47d2-a07d-0241fcda99a9 admin admin] The token cache is being invalidate because project 1368716392674f62bd5b38d607455955 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:25:49.812396 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-52deb477-9119-47d2-a07d-0241fcda99a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:49.813084 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-52deb477-9119-47d2-a07d-0241fcda99a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:49.814886 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1416/2824] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:25:49 2022] DELETE /identity/v3/projects/1368716392674f62bd5b38d607455955 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:25:49.820209 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679e57b-53ab-4208-9f3d-6ecb42d8d844 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:49.820738 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679e57b-53ab-4208-9f3d-6ecb42d8d844 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:49.821817 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679e57b-53ab-4208-9f3d-6ecb42d8d844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:49.843866 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-8679e57b-53ab-4208-9f3d-6ecb42d8d844 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:25:49.845182 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8679e57b-53ab-4208-9f3d-6ecb42d8d844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:49.922597 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1409/2825] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:25:49.925140 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb21f0-d240-40c2-b7ff-ca699ec521c6 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:49.925437 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb21f0-d240-40c2-b7ff-ca699ec521c6 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:49.925723 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb21f0-d240-40c2-b7ff-ca699ec521c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:49.947291 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-dccb21f0-d240-40c2-b7ff-ca699ec521c6 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:25:49.948540 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-dccb21f0-d240-40c2-b7ff-ca699ec521c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:49.957739 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1417/2826] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:25:49.962051 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679e57b-53ab-4208-9f3d-6ecb42d8d844 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:49.972576 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679e57b-53ab-4208-9f3d-6ecb42d8d844 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:49.978680 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077dada-193e-453d-91f9-aab5f5fc54ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:49.979409 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077dada-193e-453d-91f9-aab5f5fc54ef admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:49.979695 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077dada-193e-453d-91f9-aab5f5fc54ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:49.979972 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077dada-193e-453d-91f9-aab5f5fc54ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:49.980674 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077dada-193e-453d-91f9-aab5f5fc54ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:25:49.980978 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077dada-193e-453d-91f9-aab5f5fc54ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:49.982585 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077dada-193e-453d-91f9-aab5f5fc54ef admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:49.996213 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1410/2827] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:25:50.000684 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb21f0-d240-40c2-b7ff-ca699ec521c6 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:50.010674 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb21f0-d240-40c2-b7ff-ca699ec521c6 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:50.015697 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c95e37-4a6b-4649-9eaa-feaa71186d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:50.016389 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c95e37-4a6b-4649-9eaa-feaa71186d2a admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:50.016736 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c95e37-4a6b-4649-9eaa-feaa71186d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:50.017106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c95e37-4a6b-4649-9eaa-feaa71186d2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:50.017846 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c95e37-4a6b-4649-9eaa-feaa71186d2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:50.020006 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c95e37-4a6b-4649-9eaa-feaa71186d2a admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:50.109184 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077dada-193e-453d-91f9-aab5f5fc54ef admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:50.115650 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1418/2828] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:25:49 2022] POST /identity/v3/projects => generated 367 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:25:50.120248 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077dada-193e-453d-91f9-aab5f5fc54ef admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:50.173275 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:25:50.192614 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e1d1c1-eb69-45c3-8e60-283d0a2a7d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:50.193340 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1d1c1-eb69-45c3-8e60-283d0a2a7d55 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:50.193688 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1d1c1-eb69-45c3-8e60-283d0a2a7d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:50.194033 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1d1c1-eb69-45c3-8e60-283d0a2a7d55 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:50.194818 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e1d1c1-eb69-45c3-8e60-283d0a2a7d55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:50.196807 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e1d1c1-eb69-45c3-8e60-283d0a2a7d55 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:50.291442 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1411/2829] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:25:50 2022] POST /identity/v3/users => generated 284 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:25:50.295608 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c95e37-4a6b-4649-9eaa-feaa71186d2a admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:50.307291 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c95e37-4a6b-4649-9eaa-feaa71186d2a admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:50.312028 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da812419-2cd6-41d5-89fa-2973201c616b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:50.312878 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da812419-2cd6-41d5-89fa-2973201c616b admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:50.313003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da812419-2cd6-41d5-89fa-2973201c616b admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:50.313194 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da812419-2cd6-41d5-89fa-2973201c616b admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:50.313892 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da812419-2cd6-41d5-89fa-2973201c616b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:50.315817 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da812419-2cd6-41d5-89fa-2973201c616b admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:50.326738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1419/2830] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:25:50.330952 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e1d1c1-eb69-45c3-8e60-283d0a2a7d55 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:50.342776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e1d1c1-eb69-45c3-8e60-283d0a2a7d55 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:50.351572 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d270a-acda-4006-8228-85fd2b932d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:50.352219 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d270a-acda-4006-8228-85fd2b932d72 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:50.352504 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d270a-acda-4006-8228-85fd2b932d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:50.352751 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d270a-acda-4006-8228-85fd2b932d72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:50.353456 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d270a-acda-4006-8228-85fd2b932d72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:50.355403 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d270a-acda-4006-8228-85fd2b932d72 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:50.367983 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1412/2831] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:25:50.372008 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da812419-2cd6-41d5-89fa-2973201c616b admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:50.383275 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da812419-2cd6-41d5-89fa-2973201c616b admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:50.387977 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56771e5-3dd6-434b-8edb-ee78691a865c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:50.388638 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56771e5-3dd6-434b-8edb-ee78691a865c admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:50.388922 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56771e5-3dd6-434b-8edb-ee78691a865c admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:50.389195 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56771e5-3dd6-434b-8edb-ee78691a865c admin admin] PATH_INFO: `/v3/projects/26322ca46f724fd4b6e57e3d24acdb3f/users/001b2479944a4180802c25fc0ed6d7c0/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:50.405550 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56771e5-3dd6-434b-8edb-ee78691a865c admin admin] RBAC: Authorizing `identity:create_grant(project_id=26322ca46f724fd4b6e57e3d24acdb3f, user_id=001b2479944a4180802c25fc0ed6d7c0, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:50.408374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56771e5-3dd6-434b-8edb-ee78691a865c admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:50.456259 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1420/2832] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:25:50 2022] PUT /identity/v3/projects/26322ca46f724fd4b6e57e3d24acdb3f/users/001b2479944a4180802c25fc0ed6d7c0/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:25:50.523419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc81b8-0a36-4271-81e3-95c79f92ad17 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:50.523711 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc81b8-0a36-4271-81e3-95c79f92ad17 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:50.523993 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc81b8-0a36-4271-81e3-95c79f92ad17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:50.553904 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-c2fc81b8-0a36-4271-81e3-95c79f92ad17 None None] MFA Rules not processed for user `001b2479944a4180802c25fc0ed6d7c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:25:50.555096 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c2fc81b8-0a36-4271-81e3-95c79f92ad17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:50.621617 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1413/2833] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:25:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:25:50.626488 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56771e5-3dd6-434b-8edb-ee78691a865c admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:50.680436 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:25:50.694396 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56771e5-3dd6-434b-8edb-ee78691a865c admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:50.699463 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87eb5b6-3e60-4484-9931-81627c240395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:50.708610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87eb5b6-3e60-4484-9931-81627c240395 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:50.709006 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87eb5b6-3e60-4484-9931-81627c240395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:50.709286 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87eb5b6-3e60-4484-9931-81627c240395 admin admin] PATH_INFO: `/v3/projects` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:50.710035 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87eb5b6-3e60-4484-9931-81627c240395 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:50.712104 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87eb5b6-3e60-4484-9931-81627c240395 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:50.783860 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1421/2834] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:25:50 2022] POST /identity/v3/projects => generated 367 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:25:50.788002 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc81b8-0a36-4271-81e3-95c79f92ad17 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:50.798308 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc81b8-0a36-4271-81e3-95c79f92ad17 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:50.856547 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f5cff-d8a1-4de0-97e0-cd46fbfc6715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:50.857176 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f5cff-d8a1-4de0-97e0-cd46fbfc6715 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:50.857366 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f5cff-d8a1-4de0-97e0-cd46fbfc6715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:50.857643 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f5cff-d8a1-4de0-97e0-cd46fbfc6715 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:50.858305 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f5cff-d8a1-4de0-97e0-cd46fbfc6715 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:50.860283 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f5cff-d8a1-4de0-97e0-cd46fbfc6715 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:50.993335 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1414/2835] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:25:50 2022] POST /identity/v3/users => generated 290 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:25:50.997346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87eb5b6-3e60-4484-9931-81627c240395 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:51.008514 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87eb5b6-3e60-4484-9931-81627c240395 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:51.013075 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d18c0b1-4200-4161-b465-3de744574143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:51.013684 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18c0b1-4200-4161-b465-3de744574143 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:51.013971 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18c0b1-4200-4161-b465-3de744574143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:51.014252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18c0b1-4200-4161-b465-3de744574143 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:51.015017 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d18c0b1-4200-4161-b465-3de744574143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:51.016625 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d18c0b1-4200-4161-b465-3de744574143 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:51.026477 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1422/2836] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:25:51.030679 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f5cff-d8a1-4de0-97e0-cd46fbfc6715 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:51.041033 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f5cff-d8a1-4de0-97e0-cd46fbfc6715 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:51.045705 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4477e139-00bb-43f7-986a-b60a51a4f40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:51.046365 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477e139-00bb-43f7-986a-b60a51a4f40e admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:51.046652 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477e139-00bb-43f7-986a-b60a51a4f40e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:51.046969 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477e139-00bb-43f7-986a-b60a51a4f40e admin admin] PATH_INFO: `/v3/projects/248d363faa314322ad6630a37ef18cc3/users/52a74c262b0a429b9609ce145580f8b6/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:51.062091 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4477e139-00bb-43f7-986a-b60a51a4f40e admin admin] RBAC: Authorizing `identity:create_grant(project_id=248d363faa314322ad6630a37ef18cc3, user_id=52a74c262b0a429b9609ce145580f8b6, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:51.064847 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4477e139-00bb-43f7-986a-b60a51a4f40e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:51.129395 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1415/2837] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:25:51 2022] PUT /identity/v3/projects/248d363faa314322ad6630a37ef18cc3/users/52a74c262b0a429b9609ce145580f8b6/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:25:51.133346 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d18c0b1-4200-4161-b465-3de744574143 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:51.143386 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d18c0b1-4200-4161-b465-3de744574143 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:51.199561 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda9623-0362-4043-aa7f-85c67bfcf36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:51.200171 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda9623-0362-4043-aa7f-85c67bfcf36e admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:51.200477 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda9623-0362-4043-aa7f-85c67bfcf36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:51.200741 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda9623-0362-4043-aa7f-85c67bfcf36e admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:51.201429 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffda9623-0362-4043-aa7f-85c67bfcf36e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:51.203078 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffda9623-0362-4043-aa7f-85c67bfcf36e admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:51.212738 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1423/2838] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:25:51.216958 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4477e139-00bb-43f7-986a-b60a51a4f40e admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:51.227800 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4477e139-00bb-43f7-986a-b60a51a4f40e admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:51.232376 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4600c85-75c2-4337-b4e3-e9fd0a940708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:51.233033 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4600c85-75c2-4337-b4e3-e9fd0a940708 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:51.233332 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4600c85-75c2-4337-b4e3-e9fd0a940708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:51.233592 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4600c85-75c2-4337-b4e3-e9fd0a940708 admin admin] PATH_INFO: `/v3/projects/248d363faa314322ad6630a37ef18cc3/users/52a74c262b0a429b9609ce145580f8b6/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:51.235583 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4600c85-75c2-4337-b4e3-e9fd0a940708 admin admin] RBAC: Authorizing `identity:create_grant(project_id=248d363faa314322ad6630a37ef18cc3, user_id=52a74c262b0a429b9609ce145580f8b6, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:51.238233 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4600c85-75c2-4337-b4e3-e9fd0a940708 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:51.248837 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1416/2839] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:25:51 2022] PUT /identity/v3/projects/248d363faa314322ad6630a37ef18cc3/users/52a74c262b0a429b9609ce145580f8b6/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:25:51.258770 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97771ed-33cf-42d5-95f5-84f50b6b3851 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:51.259094 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97771ed-33cf-42d5-95f5-84f50b6b3851 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:51.259382 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97771ed-33cf-42d5-95f5-84f50b6b3851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:51.288633 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-f97771ed-33cf-42d5-95f5-84f50b6b3851 None None] MFA Rules not processed for user `52a74c262b0a429b9609ce145580f8b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:25:51.289895 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-f97771ed-33cf-42d5-95f5-84f50b6b3851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:51.359202 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1424/2840] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:25:51 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:25:51.370494 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4600c85-75c2-4337-b4e3-e9fd0a940708 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:51.372200 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a4600c85-75c2-4337-b4e3-e9fd0a940708 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:51.377731 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:25:51.430908 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4600c85-75c2-4337-b4e3-e9fd0a940708 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:51.434502 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c99401-076c-4765-b456-92c6611e76ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:51.435144 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c99401-076c-4765-b456-92c6611e76ba service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:51.435434 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c99401-076c-4765-b456-92c6611e76ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:51.435716 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c99401-076c-4765-b456-92c6611e76ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:51.445470 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c99401-076c-4765-b456-92c6611e76ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:51.446942 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c99401-076c-4765-b456-92c6611e76ba service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:51.456438 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1417/2841] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:25:51 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:25:54.076877 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97771ed-33cf-42d5-95f5-84f50b6b3851 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:54.124656 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:25:54.140048 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97771ed-33cf-42d5-95f5-84f50b6b3851 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:54.144506 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a59cf-702d-48ff-8530-f1de53d36f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:54.145767 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a59cf-702d-48ff-8530-f1de53d36f9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:54.146057 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a59cf-702d-48ff-8530-f1de53d36f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:54.146536 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a59cf-702d-48ff-8530-f1de53d36f9a admin admin] PATH_INFO: `/v3/users/001b2479944a4180802c25fc0ed6d7c0` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:54.148161 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a59cf-702d-48ff-8530-f1de53d36f9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=001b2479944a4180802c25fc0ed6d7c0)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:54.150132 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a59cf-702d-48ff-8530-f1de53d36f9a admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:54.313489 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-271a59cf-702d-48ff-8530-f1de53d36f9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '001b2479944a4180802c25fc0ed6d7c0'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:54.391370 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-271a59cf-702d-48ff-8530-f1de53d36f9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '001b2479944a4180802c25fc0ed6d7c0'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:54.392052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-271a59cf-702d-48ff-8530-f1de53d36f9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '001b2479944a4180802c25fc0ed6d7c0'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:54.405291 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-271a59cf-702d-48ff-8530-f1de53d36f9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '001b2479944a4180802c25fc0ed6d7c0'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:54.405979 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-271a59cf-702d-48ff-8530-f1de53d36f9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '001b2479944a4180802c25fc0ed6d7c0'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:54.418562 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1425/2842] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:25:54 2022] DELETE /identity/v3/users/001b2479944a4180802c25fc0ed6d7c0 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:25:54.423089 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb5132-ce93-4daa-b0a7-e1aba4c9493e None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:54.423555 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb5132-ce93-4daa-b0a7-e1aba4c9493e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:54.423679 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb5132-ce93-4daa-b0a7-e1aba4c9493e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:54.443886 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-87bb5132-ce93-4daa-b0a7-e1aba4c9493e None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:25:54.445188 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-87bb5132-ce93-4daa-b0a7-e1aba4c9493e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:54.504215 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1418/2843] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:25:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:25:54.514577 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a59cf-702d-48ff-8530-f1de53d36f9a admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:54.516058 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-271a59cf-702d-48ff-8530-f1de53d36f9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:54.574083 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a59cf-702d-48ff-8530-f1de53d36f9a admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:54.577653 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590afb28-858e-4fbf-8858-0fb19fa10397 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:54.578303 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590afb28-858e-4fbf-8858-0fb19fa10397 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:54.578557 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590afb28-858e-4fbf-8858-0fb19fa10397 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:54.578835 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590afb28-858e-4fbf-8858-0fb19fa10397 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:54.587998 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590afb28-858e-4fbf-8858-0fb19fa10397 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:54.589449 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590afb28-858e-4fbf-8858-0fb19fa10397 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:54.599156 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1426/2844] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:25:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:25:54.982369 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb5132-ce93-4daa-b0a7-e1aba4c9493e None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:54.983844 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-87bb5132-ce93-4daa-b0a7-e1aba4c9493e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:54.996377 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb5132-ce93-4daa-b0a7-e1aba4c9493e None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:55.001102 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b8537-8aad-41c4-9a18-edb4a64971d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:55.001756 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b8537-8aad-41c4-9a18-edb4a64971d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:55.002046 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b8537-8aad-41c4-9a18-edb4a64971d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:55.002319 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b8537-8aad-41c4-9a18-edb4a64971d2 admin admin] PATH_INFO: `/v3/projects/26322ca46f724fd4b6e57e3d24acdb3f` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:55.003951 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2b8537-8aad-41c4-9a18-edb4a64971d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26322ca46f724fd4b6e57e3d24acdb3f)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:55.006241 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2b8537-8aad-41c4-9a18-edb4a64971d2 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:55.094397 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-eb2b8537-8aad-41c4-9a18-edb4a64971d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26322ca46f724fd4b6e57e3d24acdb3f', 'request_id': 'req-eb2b8537-8aad-41c4-9a18-edb4a64971d2'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:55.156977 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-eb2b8537-8aad-41c4-9a18-edb4a64971d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26322ca46f724fd4b6e57e3d24acdb3f', 'request_id': 'req-eb2b8537-8aad-41c4-9a18-edb4a64971d2'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:55.163266 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-eb2b8537-8aad-41c4-9a18-edb4a64971d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26322ca46f724fd4b6e57e3d24acdb3f', 'request_id': 'req-eb2b8537-8aad-41c4-9a18-edb4a64971d2'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:55.174566 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-eb2b8537-8aad-41c4-9a18-edb4a64971d2 admin admin] The token cache is being invalidate because project 26322ca46f724fd4b6e57e3d24acdb3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:25:55.174866 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-eb2b8537-8aad-41c4-9a18-edb4a64971d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:55.175505 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-eb2b8537-8aad-41c4-9a18-edb4a64971d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:55.177614 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1419/2845] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:25:54 2022] DELETE /identity/v3/projects/26322ca46f724fd4b6e57e3d24acdb3f => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:25:55.181603 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590afb28-858e-4fbf-8858-0fb19fa10397 service neutron] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:55.191920 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590afb28-858e-4fbf-8858-0fb19fa10397 service neutron] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:55.248734 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ad4a5-8039-4974-8778-2e2cc5a66830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:55.249366 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ad4a5-8039-4974-8778-2e2cc5a66830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:55.249648 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ad4a5-8039-4974-8778-2e2cc5a66830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:55.249922 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ad4a5-8039-4974-8778-2e2cc5a66830 admin admin] PATH_INFO: `/v3/users/52a74c262b0a429b9609ce145580f8b6` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:55.251095 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ad4a5-8039-4974-8778-2e2cc5a66830 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52a74c262b0a429b9609ce145580f8b6)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:55.253013 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ad4a5-8039-4974-8778-2e2cc5a66830 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:55.262791 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:25:55.445129 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-387ad4a5-8039-4974-8778-2e2cc5a66830 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52a74c262b0a429b9609ce145580f8b6'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:55.511376 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-387ad4a5-8039-4974-8778-2e2cc5a66830 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52a74c262b0a429b9609ce145580f8b6'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:55.511996 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-387ad4a5-8039-4974-8778-2e2cc5a66830 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52a74c262b0a429b9609ce145580f8b6'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:55.525258 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-387ad4a5-8039-4974-8778-2e2cc5a66830 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52a74c262b0a429b9609ce145580f8b6'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:55.525909 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-387ad4a5-8039-4974-8778-2e2cc5a66830 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52a74c262b0a429b9609ce145580f8b6'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:55.538779 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1427/2846] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:25:55 2022] DELETE /identity/v3/users/52a74c262b0a429b9609ce145580f8b6 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:25:55.936230 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b8537-8aad-41c4-9a18-edb4a64971d2 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:55.943242 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-eb2b8537-8aad-41c4-9a18-edb4a64971d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:55.951604 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b8537-8aad-41c4-9a18-edb4a64971d2 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:55.980170 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:25:56.012272 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52612d58-bf5a-49eb-87ab-1b9a2935d3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:56.012920 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52612d58-bf5a-49eb-87ab-1b9a2935d3ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:56.013213 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52612d58-bf5a-49eb-87ab-1b9a2935d3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:56.013514 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52612d58-bf5a-49eb-87ab-1b9a2935d3ca admin admin] PATH_INFO: `/v3/projects/248d363faa314322ad6630a37ef18cc3` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:56.014994 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52612d58-bf5a-49eb-87ab-1b9a2935d3ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=248d363faa314322ad6630a37ef18cc3)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:56.017316 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52612d58-bf5a-49eb-87ab-1b9a2935d3ca admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:56.111418 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-52612d58-bf5a-49eb-87ab-1b9a2935d3ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '248d363faa314322ad6630a37ef18cc3', 'request_id': 'req-52612d58-bf5a-49eb-87ab-1b9a2935d3ca'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:56.179575 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-52612d58-bf5a-49eb-87ab-1b9a2935d3ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '248d363faa314322ad6630a37ef18cc3', 'request_id': 'req-52612d58-bf5a-49eb-87ab-1b9a2935d3ca'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:56.185523 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-52612d58-bf5a-49eb-87ab-1b9a2935d3ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '248d363faa314322ad6630a37ef18cc3', 'request_id': 'req-52612d58-bf5a-49eb-87ab-1b9a2935d3ca'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:56.198195 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-52612d58-bf5a-49eb-87ab-1b9a2935d3ca admin admin] The token cache is being invalidate because project 248d363faa314322ad6630a37ef18cc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:25:56.198489 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-52612d58-bf5a-49eb-87ab-1b9a2935d3ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:56.199154 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-52612d58-bf5a-49eb-87ab-1b9a2935d3ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:25:56.200664 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1420/2847] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:25:55 2022] DELETE /identity/v3/projects/248d363faa314322ad6630a37ef18cc3 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:25:56.205731 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbc0d8-ef3d-4f49-81f8-6d35c1c89880 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:56.206213 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbc0d8-ef3d-4f49-81f8-6d35c1c89880 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:56.206325 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbc0d8-ef3d-4f49-81f8-6d35c1c89880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:56.225589 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-05bbc0d8-ef3d-4f49-81f8-6d35c1c89880 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:25:56.226823 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-05bbc0d8-ef3d-4f49-81f8-6d35c1c89880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:56.290545 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1428/2848] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:25:56.299590 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086526e-b376-4b8f-b7a1-243f45a732c6 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:56.299881 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086526e-b376-4b8f-b7a1-243f45a732c6 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:56.300162 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086526e-b376-4b8f-b7a1-243f45a732c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:56.321292 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-b086526e-b376-4b8f-b7a1-243f45a732c6 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:25:56.322501 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b086526e-b376-4b8f-b7a1-243f45a732c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:56.330439 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1421/2849] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:25:56.334643 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bbc0d8-ef3d-4f49-81f8-6d35c1c89880 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:56.344310 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bbc0d8-ef3d-4f49-81f8-6d35c1c89880 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:56.349318 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c874c-2db6-4027-82f1-3c2e5fd4048b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:56.349917 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c874c-2db6-4027-82f1-3c2e5fd4048b admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:56.350195 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c874c-2db6-4027-82f1-3c2e5fd4048b admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:56.350469 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c874c-2db6-4027-82f1-3c2e5fd4048b admin admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:56.351195 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c874c-2db6-4027-82f1-3c2e5fd4048b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:25:56.351499 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c874c-2db6-4027-82f1-3c2e5fd4048b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:56.353095 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c874c-2db6-4027-82f1-3c2e5fd4048b admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:56.365026 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1429/2850] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 05:25:56.369295 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b086526e-b376-4b8f-b7a1-243f45a732c6 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:56.378574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b086526e-b376-4b8f-b7a1-243f45a732c6 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:56.383282 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588a8f2-2ed0-4011-a103-6b57c0a6225c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:56.383885 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588a8f2-2ed0-4011-a103-6b57c0a6225c admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:56.384169 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588a8f2-2ed0-4011-a103-6b57c0a6225c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:56.384444 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588a8f2-2ed0-4011-a103-6b57c0a6225c admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:56.385174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588a8f2-2ed0-4011-a103-6b57c0a6225c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:56.387152 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588a8f2-2ed0-4011-a103-6b57c0a6225c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:56.485341 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1422/2851] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:25:56 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:25:56.489720 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c874c-2db6-4027-82f1-3c2e5fd4048b admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:56.500093 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c874c-2db6-4027-82f1-3c2e5fd4048b admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:56.558954 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af53c25d-a207-4001-a3eb-a26089cf6ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:56.559237 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53c25d-a207-4001-a3eb-a26089cf6ab2 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:56.559520 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53c25d-a207-4001-a3eb-a26089cf6ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:56.559795 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53c25d-a207-4001-a3eb-a26089cf6ab2 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:56.560473 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af53c25d-a207-4001-a3eb-a26089cf6ab2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:56.562401 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af53c25d-a207-4001-a3eb-a26089cf6ab2 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:56.637757 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1430/2852] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:25:56 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:25:56.642046 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588a8f2-2ed0-4011-a103-6b57c0a6225c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:56.652658 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588a8f2-2ed0-4011-a103-6b57c0a6225c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:56.657343 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cdfde7-2f2c-412b-8af5-81ea7dd693be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:56.657959 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cdfde7-2f2c-412b-8af5-81ea7dd693be admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:56.658241 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cdfde7-2f2c-412b-8af5-81ea7dd693be admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:56.658513 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cdfde7-2f2c-412b-8af5-81ea7dd693be admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:56.659245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cdfde7-2f2c-412b-8af5-81ea7dd693be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:56.661355 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cdfde7-2f2c-412b-8af5-81ea7dd693be admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:56.674772 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1423/2853] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:25:56.676110 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af53c25d-a207-4001-a3eb-a26089cf6ab2 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:56.683184 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:25:56.687052 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af53c25d-a207-4001-a3eb-a26089cf6ab2 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:56.691820 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0a4ef-262f-4410-8672-8197c7eb3bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:56.692467 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0a4ef-262f-4410-8672-8197c7eb3bf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:56.692750 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0a4ef-262f-4410-8672-8197c7eb3bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:56.693026 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0a4ef-262f-4410-8672-8197c7eb3bf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:56.693760 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc0a4ef-262f-4410-8672-8197c7eb3bf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:56.695373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc0a4ef-262f-4410-8672-8197c7eb3bf3 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:56.705879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1431/2854] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:25:56.710019 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cdfde7-2f2c-412b-8af5-81ea7dd693be admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:56.720139 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cdfde7-2f2c-412b-8af5-81ea7dd693be admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:56.725460 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747d6cf-ee2b-4b66-8adb-6dfd16eddb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:56.726108 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747d6cf-ee2b-4b66-8adb-6dfd16eddb6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:56.726393 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747d6cf-ee2b-4b66-8adb-6dfd16eddb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:56.726669 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747d6cf-ee2b-4b66-8adb-6dfd16eddb6c admin admin] PATH_INFO: `/v3/projects/ba29601fd6f646b59d625a5825afc2c1/users/557ade07125748fb92634a691a6dae6f/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:56.742111 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747d6cf-ee2b-4b66-8adb-6dfd16eddb6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba29601fd6f646b59d625a5825afc2c1, user_id=557ade07125748fb92634a691a6dae6f, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:56.746162 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747d6cf-ee2b-4b66-8adb-6dfd16eddb6c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:56.815096 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1424/2855] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:25:56 2022] PUT /identity/v3/projects/ba29601fd6f646b59d625a5825afc2c1/users/557ade07125748fb92634a691a6dae6f/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:25:56.825649 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057b787-59d5-4f31-8c59-b1b49eb6a0c0 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:56.834678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057b787-59d5-4f31-8c59-b1b49eb6a0c0 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:56.834678 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057b787-59d5-4f31-8c59-b1b49eb6a0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:56.855460 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-5057b787-59d5-4f31-8c59-b1b49eb6a0c0 None None] MFA Rules not processed for user `557ade07125748fb92634a691a6dae6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:25:56.856728 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-5057b787-59d5-4f31-8c59-b1b49eb6a0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:56.923394 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1432/2856] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:25:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:25:56.927915 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747d6cf-ee2b-4b66-8adb-6dfd16eddb6c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:56.984986 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:25:56.989995 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747d6cf-ee2b-4b66-8adb-6dfd16eddb6c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:56.994735 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c48785-e7d3-4052-9092-0e84f42c4aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:56.998469 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c48785-e7d3-4052-9092-0e84f42c4aa4 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:56.998469 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c48785-e7d3-4052-9092-0e84f42c4aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:56.998469 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c48785-e7d3-4052-9092-0e84f42c4aa4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:56.998469 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c48785-e7d3-4052-9092-0e84f42c4aa4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:56.998875 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c48785-e7d3-4052-9092-0e84f42c4aa4 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:57.102725 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1425/2857] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:25:56 2022] POST /identity/v3/projects => generated 385 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:25:57.107032 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057b787-59d5-4f31-8c59-b1b49eb6a0c0 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:57.117989 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057b787-59d5-4f31-8c59-b1b49eb6a0c0 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:57.173145 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba469f-6cee-49e4-96ca-3256d730bedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:57.173752 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba469f-6cee-49e4-96ca-3256d730bedc admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:57.174033 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba469f-6cee-49e4-96ca-3256d730bedc admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:57.174304 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba469f-6cee-49e4-96ca-3256d730bedc admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:57.175003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ba469f-6cee-49e4-96ca-3256d730bedc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:57.176928 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ba469f-6cee-49e4-96ca-3256d730bedc admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:57.249795 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1433/2858] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:25:57 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:25:57.254978 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c48785-e7d3-4052-9092-0e84f42c4aa4 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:57.265337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c48785-e7d3-4052-9092-0e84f42c4aa4 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:57.270680 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3bd58-c0ab-4bf3-94d9-54a2212054c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:57.271325 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3bd58-c0ab-4bf3-94d9-54a2212054c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:57.271611 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3bd58-c0ab-4bf3-94d9-54a2212054c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:57.271888 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3bd58-c0ab-4bf3-94d9-54a2212054c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:57.272580 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc3bd58-c0ab-4bf3-94d9-54a2212054c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:57.274181 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc3bd58-c0ab-4bf3-94d9-54a2212054c9 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:57.284268 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1426/2859] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:25:57.288295 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba469f-6cee-49e4-96ca-3256d730bedc admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:57.298439 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba469f-6cee-49e4-96ca-3256d730bedc admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:57.303023 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208ec89-ded4-43dc-8aa4-70af8f7662a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:57.303623 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208ec89-ded4-43dc-8aa4-70af8f7662a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:57.303903 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208ec89-ded4-43dc-8aa4-70af8f7662a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:57.304177 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208ec89-ded4-43dc-8aa4-70af8f7662a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:57.304863 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208ec89-ded4-43dc-8aa4-70af8f7662a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:57.306456 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208ec89-ded4-43dc-8aa4-70af8f7662a4 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:57.316491 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1434/2860] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:25:57.321015 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3bd58-c0ab-4bf3-94d9-54a2212054c9 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:57.331016 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3bd58-c0ab-4bf3-94d9-54a2212054c9 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:57.336258 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2608e-be48-4546-80cd-6330b4ef0d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:57.336913 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2608e-be48-4546-80cd-6330b4ef0d4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:57.337200 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2608e-be48-4546-80cd-6330b4ef0d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:57.337483 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2608e-be48-4546-80cd-6330b4ef0d4a admin admin] PATH_INFO: `/v3/projects/a0896c55a5844de09e03305a8d6a7757/users/dcb965b3d47947f0866b57ab97d58106/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:57.352695 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d2608e-be48-4546-80cd-6330b4ef0d4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0896c55a5844de09e03305a8d6a7757, user_id=dcb965b3d47947f0866b57ab97d58106, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:57.355484 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d2608e-be48-4546-80cd-6330b4ef0d4a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:57.415044 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1427/2861] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:25:57 2022] PUT /identity/v3/projects/a0896c55a5844de09e03305a8d6a7757/users/dcb965b3d47947f0866b57ab97d58106/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 05:25:57.432647 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab86031-3dc9-4e90-a4de-40d8a9eee553 None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:57.432943 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab86031-3dc9-4e90-a4de-40d8a9eee553 None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:57.433228 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab86031-3dc9-4e90-a4de-40d8a9eee553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:57.462455 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-aab86031-3dc9-4e90-a4de-40d8a9eee553 None None] MFA Rules not processed for user `dcb965b3d47947f0866b57ab97d58106`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:25:57.463694 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-aab86031-3dc9-4e90-a4de-40d8a9eee553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:57.511981 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:25:57.529354 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1435/2862] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:25:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 05:25:57.534012 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2608e-be48-4546-80cd-6330b4ef0d4a admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:57.598028 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2608e-be48-4546-80cd-6330b4ef0d4a admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:57.604149 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d96bb2-f64f-4caa-8ea0-dc07910cdb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:57.604763 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d96bb2-f64f-4caa-8ea0-dc07910cdb0f admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:57.605050 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d96bb2-f64f-4caa-8ea0-dc07910cdb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:57.605325 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d96bb2-f64f-4caa-8ea0-dc07910cdb0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:57.606160 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d96bb2-f64f-4caa-8ea0-dc07910cdb0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:57.608205 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d96bb2-f64f-4caa-8ea0-dc07910cdb0f admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:57.701905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1428/2863] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:25:57 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:25:57.706218 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab86031-3dc9-4e90-a4de-40d8a9eee553 None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:57.717599 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab86031-3dc9-4e90-a4de-40d8a9eee553 None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:57.773521 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d6b8e-2a1c-4f93-ab47-1970581bf0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:57.774160 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d6b8e-2a1c-4f93-ab47-1970581bf0f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:57.774441 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d6b8e-2a1c-4f93-ab47-1970581bf0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:57.774717 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d6b8e-2a1c-4f93-ab47-1970581bf0f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:57.775476 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d6b8e-2a1c-4f93-ab47-1970581bf0f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:57.777444 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d6b8e-2a1c-4f93-ab47-1970581bf0f3 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:57.850014 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1436/2864] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:25:57 2022] POST /identity/v3/users => generated 299 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:25:57.854218 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d96bb2-f64f-4caa-8ea0-dc07910cdb0f admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:57.864760 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d96bb2-f64f-4caa-8ea0-dc07910cdb0f admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:57.869302 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cbb42c-420f-481e-a833-24dcd6e5d19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:57.869912 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbb42c-420f-481e-a833-24dcd6e5d19a admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:57.870197 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbb42c-420f-481e-a833-24dcd6e5d19a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:57.870473 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbb42c-420f-481e-a833-24dcd6e5d19a admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:57.871196 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cbb42c-420f-481e-a833-24dcd6e5d19a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:57.872798 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cbb42c-420f-481e-a833-24dcd6e5d19a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:57.884127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1429/2865] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:25:57.888490 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d6b8e-2a1c-4f93-ab47-1970581bf0f3 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:57.899275 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d6b8e-2a1c-4f93-ab47-1970581bf0f3 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:57.903994 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a178cc4-00a0-409b-918d-f057773834f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:57.904689 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a178cc4-00a0-409b-918d-f057773834f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:57.904974 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a178cc4-00a0-409b-918d-f057773834f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:57.905252 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a178cc4-00a0-409b-918d-f057773834f1 admin admin] PATH_INFO: `/v3/projects/f742246195be47328606dca4c3afff10/users/65d96692f006409489bea8d3612a66fc/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:57.919731 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a178cc4-00a0-409b-918d-f057773834f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f742246195be47328606dca4c3afff10, user_id=65d96692f006409489bea8d3612a66fc, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:57.922628 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a178cc4-00a0-409b-918d-f057773834f1 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:57.991118 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1437/2866] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:25:57 2022] PUT /identity/v3/projects/f742246195be47328606dca4c3afff10/users/65d96692f006409489bea8d3612a66fc/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:25:57.995136 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cbb42c-420f-481e-a833-24dcd6e5d19a admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:58.005856 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cbb42c-420f-481e-a833-24dcd6e5d19a admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:58.061570 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3632042-5e0b-41bc-8824-b59bff516093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:58.062186 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3632042-5e0b-41bc-8824-b59bff516093 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:58.062497 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3632042-5e0b-41bc-8824-b59bff516093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:58.062777 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3632042-5e0b-41bc-8824-b59bff516093 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:58.063518 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3632042-5e0b-41bc-8824-b59bff516093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:58.065523 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3632042-5e0b-41bc-8824-b59bff516093 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:58.076159 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1430/2867] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:25:58.080329 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a178cc4-00a0-409b-918d-f057773834f1 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:58.091513 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a178cc4-00a0-409b-918d-f057773834f1 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:58.096357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e131f4-7f03-41a4-9796-392f81511836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:58.097016 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e131f4-7f03-41a4-9796-392f81511836 admin admin] REQUEST_METHOD: `PUT` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:58.097297 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e131f4-7f03-41a4-9796-392f81511836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:58.097573 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e131f4-7f03-41a4-9796-392f81511836 admin admin] PATH_INFO: `/v3/projects/f742246195be47328606dca4c3afff10/users/65d96692f006409489bea8d3612a66fc/roles/e39d4ecea4c64d5794e1079c9be3ad36` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:58.099750 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e131f4-7f03-41a4-9796-392f81511836 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f742246195be47328606dca4c3afff10, user_id=65d96692f006409489bea8d3612a66fc, role_id=e39d4ecea4c64d5794e1079c9be3ad36)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:58.102400 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e131f4-7f03-41a4-9796-392f81511836 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:58.112600 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1438/2868] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:25:58 2022] PUT /identity/v3/projects/f742246195be47328606dca4c3afff10/users/65d96692f006409489bea8d3612a66fc/roles/e39d4ecea4c64d5794e1079c9be3ad36 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:25:58.122634 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94c6dd-a055-49ba-84f1-afdbc20e7f6a None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:58.122894 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94c6dd-a055-49ba-84f1-afdbc20e7f6a None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:58.123181 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94c6dd-a055-49ba-84f1-afdbc20e7f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:58.151632 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-cd94c6dd-a055-49ba-84f1-afdbc20e7f6a None None] MFA Rules not processed for user `65d96692f006409489bea8d3612a66fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:25:58.152844 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-cd94c6dd-a055-49ba-84f1-afdbc20e7f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:58.156936 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:25:58.222985 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1431/2869] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:25:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:25:58.233967 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e131f4-7f03-41a4-9796-392f81511836 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:58.235775 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-87e131f4-7f03-41a4-9796-392f81511836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:58.305772 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e131f4-7f03-41a4-9796-392f81511836 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:58.309517 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf43e70-1eb9-4fca-8d01-c56678454876 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:58.310186 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf43e70-1eb9-4fca-8d01-c56678454876 service cinder] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:58.310472 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf43e70-1eb9-4fca-8d01-c56678454876 service cinder] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:58.310750 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf43e70-1eb9-4fca-8d01-c56678454876 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:58.320661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf43e70-1eb9-4fca-8d01-c56678454876 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:58.322137 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf43e70-1eb9-4fca-8d01-c56678454876 service cinder] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:58.331806 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1439/2870] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:25:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:25:59.352882 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94c6dd-a055-49ba-84f1-afdbc20e7f6a None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:25:59.354281 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-cd94c6dd-a055-49ba-84f1-afdbc20e7f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:25:59.367840 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94c6dd-a055-49ba-84f1-afdbc20e7f6a None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:25:59.371679 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325af1b-1dd6-43bb-a240-6c771bab06f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:25:59.372297 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325af1b-1dd6-43bb-a240-6c771bab06f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:25:59.372585 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325af1b-1dd6-43bb-a240-6c771bab06f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:25:59.372866 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325af1b-1dd6-43bb-a240-6c771bab06f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:25:59.429849 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325af1b-1dd6-43bb-a240-6c771bab06f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:25:59.431333 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325af1b-1dd6-43bb-a240-6c771bab06f9 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:25:59.440473 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1432/2871] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:25:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:26:02.864516 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf43e70-1eb9-4fca-8d01-c56678454876 service cinder] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:26:02.869003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-aaf43e70-1eb9-4fca-8d01-c56678454876 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:26:02.918745 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:26:02.940373 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf43e70-1eb9-4fca-8d01-c56678454876 service cinder] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:26:02.945451 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d500b06-7376-4226-84f2-1a9f4db30b3d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59846d88f4314c2382aa5cb8bae5053b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:26:02.946069 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d500b06-7376-4226-84f2-1a9f4db30b3d service placement] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:26:02.946357 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d500b06-7376-4226-84f2-1a9f4db30b3d service placement] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:26:02.946639 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d500b06-7376-4226-84f2-1a9f4db30b3d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:26:02.948493 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-4d500b06-7376-4226-84f2-1a9f4db30b3d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:26:02.961190 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d500b06-7376-4226-84f2-1a9f4db30b3d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:26:02.962780 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d500b06-7376-4226-84f2-1a9f4db30b3d service placement] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:26:02.972675 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1440/2872] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:26:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:26:04.126648 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325af1b-1dd6-43bb-a240-6c771bab06f9 service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:26:04.136826 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325af1b-1dd6-43bb-a240-6c771bab06f9 service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:26:04.140755 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31047d1a-a113-41ba-aa23-6480add53c6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:26:04.141396 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31047d1a-a113-41ba-aa23-6480add53c6f service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:26:04.141683 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31047d1a-a113-41ba-aa23-6480add53c6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:26:04.141963 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31047d1a-a113-41ba-aa23-6480add53c6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:26:04.199262 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31047d1a-a113-41ba-aa23-6480add53c6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:26:04.200688 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31047d1a-a113-41ba-aa23-6480add53c6f service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:26:04.210331 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1433/2873] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:26:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:26:07.913002 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d500b06-7376-4226-84f2-1a9f4db30b3d service placement] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:26:07.975767 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d500b06-7376-4226-84f2-1a9f4db30b3d service placement] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:26:07.980335 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d101d2f-bd45-4563-b77b-0bc6ee87a19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:26:07.980984 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d101d2f-bd45-4563-b77b-0bc6ee87a19f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:26:07.981272 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d101d2f-bd45-4563-b77b-0bc6ee87a19f admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:26:07.981557 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d101d2f-bd45-4563-b77b-0bc6ee87a19f admin admin] PATH_INFO: `/v3/users/557ade07125748fb92634a691a6dae6f` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:26:07.982736 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d101d2f-bd45-4563-b77b-0bc6ee87a19f admin admin] RBAC: Authorizing `identity:delete_user(user_id=557ade07125748fb92634a691a6dae6f)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:26:07.984719 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d101d2f-bd45-4563-b77b-0bc6ee87a19f admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:26:08.188450 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-5d101d2f-bd45-4563-b77b-0bc6ee87a19f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '557ade07125748fb92634a691a6dae6f'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:08.264629 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-5d101d2f-bd45-4563-b77b-0bc6ee87a19f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '557ade07125748fb92634a691a6dae6f'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:08.268204 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-5d101d2f-bd45-4563-b77b-0bc6ee87a19f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '557ade07125748fb92634a691a6dae6f'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:08.278503 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-5d101d2f-bd45-4563-b77b-0bc6ee87a19f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '557ade07125748fb92634a691a6dae6f'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:08.279164 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-5d101d2f-bd45-4563-b77b-0bc6ee87a19f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '557ade07125748fb92634a691a6dae6f'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:08.292278 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1441/2874] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:26:07 2022] DELETE /identity/v3/users/557ade07125748fb92634a691a6dae6f => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:26:08.296992 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0dd65a-db40-4334-8a3d-5e94a6157475 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:26:08.297479 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0dd65a-db40-4334-8a3d-5e94a6157475 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:26:08.297591 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0dd65a-db40-4334-8a3d-5e94a6157475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:26:08.316530 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-5c0dd65a-db40-4334-8a3d-5e94a6157475 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:26:08.317720 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-5c0dd65a-db40-4334-8a3d-5e94a6157475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:26:08.353823 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:26:08.380887 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1434/2875] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:26:08.391079 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d101d2f-bd45-4563-b77b-0bc6ee87a19f admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:26:08.392516 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-5d101d2f-bd45-4563-b77b-0bc6ee87a19f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:26:08.449454 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d101d2f-bd45-4563-b77b-0bc6ee87a19f admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:26:08.453148 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9b6ec-8356-4585-98ad-85ac421e159b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:26:08.453790 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9b6ec-8356-4585-98ad-85ac421e159b service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:26:08.454075 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9b6ec-8356-4585-98ad-85ac421e159b service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:26:08.454353 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9b6ec-8356-4585-98ad-85ac421e159b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:26:08.463524 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9b6ec-8356-4585-98ad-85ac421e159b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:26:08.465007 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9b6ec-8356-4585-98ad-85ac421e159b service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:26:08.469051 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:26:08.475879 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1442/2876] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:26:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:26:09.153528 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0dd65a-db40-4334-8a3d-5e94a6157475 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:26:09.154977 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-5c0dd65a-db40-4334-8a3d-5e94a6157475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:26:09.166725 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0dd65a-db40-4334-8a3d-5e94a6157475 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:26:09.171366 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eadafd-e443-4a5f-927d-49db37a50015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:26:09.172013 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eadafd-e443-4a5f-927d-49db37a50015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:26:09.172321 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eadafd-e443-4a5f-927d-49db37a50015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:26:09.172667 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eadafd-e443-4a5f-927d-49db37a50015 admin admin] PATH_INFO: `/v3/projects/ba29601fd6f646b59d625a5825afc2c1` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:26:09.175127 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eadafd-e443-4a5f-927d-49db37a50015 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba29601fd6f646b59d625a5825afc2c1)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:26:09.175785 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eadafd-e443-4a5f-927d-49db37a50015 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:26:09.272371 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-32eadafd-e443-4a5f-927d-49db37a50015 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba29601fd6f646b59d625a5825afc2c1', 'request_id': 'req-32eadafd-e443-4a5f-927d-49db37a50015'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:09.351369 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-32eadafd-e443-4a5f-927d-49db37a50015 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba29601fd6f646b59d625a5825afc2c1', 'request_id': 'req-32eadafd-e443-4a5f-927d-49db37a50015'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:09.357474 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-32eadafd-e443-4a5f-927d-49db37a50015 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba29601fd6f646b59d625a5825afc2c1', 'request_id': 'req-32eadafd-e443-4a5f-927d-49db37a50015'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:09.368614 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-32eadafd-e443-4a5f-927d-49db37a50015 admin admin] The token cache is being invalidate because project ba29601fd6f646b59d625a5825afc2c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:26:09.368907 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-32eadafd-e443-4a5f-927d-49db37a50015 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:09.369539 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-32eadafd-e443-4a5f-927d-49db37a50015 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:09.371117 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1435/2877] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:26:09 2022] DELETE /identity/v3/projects/ba29601fd6f646b59d625a5825afc2c1 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:26:09.375155 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9b6ec-8356-4585-98ad-85ac421e159b service neutron] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:26:09.385591 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9b6ec-8356-4585-98ad-85ac421e159b service neutron] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:26:09.444175 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9e3e7-ac83-43c5-a9a5-196e80234b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:26:09.445437 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9e3e7-ac83-43c5-a9a5-196e80234b34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:26:09.445724 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9e3e7-ac83-43c5-a9a5-196e80234b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:26:09.446003 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9e3e7-ac83-43c5-a9a5-196e80234b34 admin admin] PATH_INFO: `/v3/users/dcb965b3d47947f0866b57ab97d58106` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:26:09.447212 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c9e3e7-ac83-43c5-a9a5-196e80234b34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcb965b3d47947f0866b57ab97d58106)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:26:09.449181 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c9e3e7-ac83-43c5-a9a5-196e80234b34 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:26:09.610382 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-28c9e3e7-ac83-43c5-a9a5-196e80234b34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcb965b3d47947f0866b57ab97d58106'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:09.676951 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-28c9e3e7-ac83-43c5-a9a5-196e80234b34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcb965b3d47947f0866b57ab97d58106'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:09.677596 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-28c9e3e7-ac83-43c5-a9a5-196e80234b34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcb965b3d47947f0866b57ab97d58106'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:09.691130 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-28c9e3e7-ac83-43c5-a9a5-196e80234b34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcb965b3d47947f0866b57ab97d58106'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:09.691769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-28c9e3e7-ac83-43c5-a9a5-196e80234b34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcb965b3d47947f0866b57ab97d58106'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:09.704791 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1443/2878] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:26:09 2022] DELETE /identity/v3/users/dcb965b3d47947f0866b57ab97d58106 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:26:10.095541 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eadafd-e443-4a5f-927d-49db37a50015 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:26:10.096981 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-32eadafd-e443-4a5f-927d-49db37a50015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:26:10.110655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eadafd-e443-4a5f-927d-49db37a50015 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:26:10.172375 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ea47d-1b79-44a3-afba-0c8d316871ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:26:10.172941 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ea47d-1b79-44a3-afba-0c8d316871ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:26:10.173227 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ea47d-1b79-44a3-afba-0c8d316871ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:26:10.173512 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ea47d-1b79-44a3-afba-0c8d316871ab admin admin] PATH_INFO: `/v3/projects/a0896c55a5844de09e03305a8d6a7757` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:26:10.174707 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ea47d-1b79-44a3-afba-0c8d316871ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0896c55a5844de09e03305a8d6a7757)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:26:10.176655 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ea47d-1b79-44a3-afba-0c8d316871ab admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:26:10.271923 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5f3ea47d-1b79-44a3-afba-0c8d316871ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0896c55a5844de09e03305a8d6a7757', 'request_id': 'req-5f3ea47d-1b79-44a3-afba-0c8d316871ab'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:10.322578 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5f3ea47d-1b79-44a3-afba-0c8d316871ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0896c55a5844de09e03305a8d6a7757', 'request_id': 'req-5f3ea47d-1b79-44a3-afba-0c8d316871ab'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:10.327701 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:26:10.328929 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5f3ea47d-1b79-44a3-afba-0c8d316871ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0896c55a5844de09e03305a8d6a7757', 'request_id': 'req-5f3ea47d-1b79-44a3-afba-0c8d316871ab'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:10.344854 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5f3ea47d-1b79-44a3-afba-0c8d316871ab admin admin] The token cache is being invalidate because project a0896c55a5844de09e03305a8d6a7757 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:26:10.345389 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5f3ea47d-1b79-44a3-afba-0c8d316871ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:10.346510 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5f3ea47d-1b79-44a3-afba-0c8d316871ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:10.349256 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1436/2879] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:26:10 2022] DELETE /identity/v3/projects/a0896c55a5844de09e03305a8d6a7757 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:26:10.355125 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9e3e7-ac83-43c5-a9a5-196e80234b34 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:26:10.359076 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-28c9e3e7-ac83-43c5-a9a5-196e80234b34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:26:10.370372 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9e3e7-ac83-43c5-a9a5-196e80234b34 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:26:10.426874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83327b5e-e77d-47fe-9912-8a5b7ff1bc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:26:10.427517 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83327b5e-e77d-47fe-9912-8a5b7ff1bc87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:26:10.427715 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83327b5e-e77d-47fe-9912-8a5b7ff1bc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:26:10.427996 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83327b5e-e77d-47fe-9912-8a5b7ff1bc87 admin admin] PATH_INFO: `/v3/users/65d96692f006409489bea8d3612a66fc` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:26:10.429729 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83327b5e-e77d-47fe-9912-8a5b7ff1bc87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65d96692f006409489bea8d3612a66fc)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:26:10.431729 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83327b5e-e77d-47fe-9912-8a5b7ff1bc87 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:26:10.453140 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:26:10.662731 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-83327b5e-e77d-47fe-9912-8a5b7ff1bc87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65d96692f006409489bea8d3612a66fc'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:10.739725 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-83327b5e-e77d-47fe-9912-8a5b7ff1bc87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65d96692f006409489bea8d3612a66fc'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:10.740294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-83327b5e-e77d-47fe-9912-8a5b7ff1bc87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65d96692f006409489bea8d3612a66fc'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:10.753833 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-83327b5e-e77d-47fe-9912-8a5b7ff1bc87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65d96692f006409489bea8d3612a66fc'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:10.754495 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-83327b5e-e77d-47fe-9912-8a5b7ff1bc87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65d96692f006409489bea8d3612a66fc'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:10.766964 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1444/2880] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:26:10 2022] DELETE /identity/v3/users/65d96692f006409489bea8d3612a66fc => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:26:11.204574 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ea47d-1b79-44a3-afba-0c8d316871ab admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:26:11.206229 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-5f3ea47d-1b79-44a3-afba-0c8d316871ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:26:11.220582 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ea47d-1b79-44a3-afba-0c8d316871ab admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:26:11.279945 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0260fb-d454-47ea-afe5-773d6b7721e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:26:11.280516 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0260fb-d454-47ea-afe5-773d6b7721e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:26:11.280801 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0260fb-d454-47ea-afe5-773d6b7721e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:26:11.281086 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0260fb-d454-47ea-afe5-773d6b7721e9 admin admin] PATH_INFO: `/v3/projects/f742246195be47328606dca4c3afff10` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:26:11.282250 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0260fb-d454-47ea-afe5-773d6b7721e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f742246195be47328606dca4c3afff10)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:26:11.284202 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0260fb-d454-47ea-afe5-773d6b7721e9 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:26:11.394634 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ec0260fb-d454-47ea-afe5-773d6b7721e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f742246195be47328606dca4c3afff10', 'request_id': 'req-ec0260fb-d454-47ea-afe5-773d6b7721e9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:11.505589 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ec0260fb-d454-47ea-afe5-773d6b7721e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f742246195be47328606dca4c3afff10', 'request_id': 'req-ec0260fb-d454-47ea-afe5-773d6b7721e9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:11.512253 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ec0260fb-d454-47ea-afe5-773d6b7721e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f742246195be47328606dca4c3afff10', 'request_id': 'req-ec0260fb-d454-47ea-afe5-773d6b7721e9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:11.524034 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ec0260fb-d454-47ea-afe5-773d6b7721e9 admin admin] The token cache is being invalidate because project f742246195be47328606dca4c3afff10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:26:11.524325 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ec0260fb-d454-47ea-afe5-773d6b7721e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:11.524961 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ec0260fb-d454-47ea-afe5-773d6b7721e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:11.526481 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1437/2881] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:26:11 2022] DELETE /identity/v3/projects/f742246195be47328606dca4c3afff10 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:26:11.533816 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ab908-2735-4400-9d74-d39f374f446b None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:26:11.534299 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ab908-2735-4400-9d74-d39f374f446b None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:26:11.534299 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ab908-2735-4400-9d74-d39f374f446b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:26:11.553769 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-796ab908-2735-4400-9d74-d39f374f446b None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:26:11.555748 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-796ab908-2735-4400-9d74-d39f374f446b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:26:11.616027 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1445/2882] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:26:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:26:11.624935 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f07b8-6536-4258-a6eb-5e730f3b2cca None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:26:11.625235 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f07b8-6536-4258-a6eb-5e730f3b2cca None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:26:11.625519 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f07b8-6536-4258-a6eb-5e730f3b2cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:26:11.644939 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-122f07b8-6536-4258-a6eb-5e730f3b2cca None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:26:11.646152 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-122f07b8-6536-4258-a6eb-5e730f3b2cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:26:11.654670 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1438/2883] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:26:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:26:11.658978 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ab908-2735-4400-9d74-d39f374f446b None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:26:11.669436 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ab908-2735-4400-9d74-d39f374f446b None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:26:11.674024 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439eda4-32fd-4356-acca-ae9e2d10df9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:26:11.674635 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439eda4-32fd-4356-acca-ae9e2d10df9c admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:26:11.674954 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439eda4-32fd-4356-acca-ae9e2d10df9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:26:11.675285 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439eda4-32fd-4356-acca-ae9e2d10df9c admin admin] PATH_INFO: `/v3/domains` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:26:11.675985 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1439eda4-32fd-4356-acca-ae9e2d10df9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80356) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 05:26:11.676293 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1439eda4-32fd-4356-acca-ae9e2d10df9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:26:11.677896 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1439eda4-32fd-4356-acca-ae9e2d10df9c admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:26:11.689881 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1446/2884] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Aug 3 05:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 05:26:11.694122 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f07b8-6536-4258-a6eb-5e730f3b2cca None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:26:11.703439 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f07b8-6536-4258-a6eb-5e730f3b2cca None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:26:11.708114 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa67a44-a54f-4e4e-a578-d6ce045aa1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:26:11.708718 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa67a44-a54f-4e4e-a578-d6ce045aa1be admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:26:11.709004 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa67a44-a54f-4e4e-a578-d6ce045aa1be admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:26:11.709283 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa67a44-a54f-4e4e-a578-d6ce045aa1be admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:26:11.709960 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa67a44-a54f-4e4e-a578-d6ce045aa1be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:26:11.712411 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa67a44-a54f-4e4e-a578-d6ce045aa1be admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:26:11.856231 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1439/2885] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Aug 3 05:26:11 2022] POST /identity/v3/projects => generated 363 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 05:26:11.860458 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439eda4-32fd-4356-acca-ae9e2d10df9c admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:26:11.870633 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439eda4-32fd-4356-acca-ae9e2d10df9c admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:26:11.926764 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e2a29-7186-4236-8fe5-cd3fc4aa7f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:26:11.927418 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e2a29-7186-4236-8fe5-cd3fc4aa7f8a admin admin] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:26:11.927707 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e2a29-7186-4236-8fe5-cd3fc4aa7f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:26:11.927990 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e2a29-7186-4236-8fe5-cd3fc4aa7f8a admin admin] PATH_INFO: `/v3/users` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:26:11.928667 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e2a29-7186-4236-8fe5-cd3fc4aa7f8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:26:11.931259 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e2a29-7186-4236-8fe5-cd3fc4aa7f8a admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:26:12.025409 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1447/2886] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Aug 3 05:26:11 2022] POST /identity/v3/users => generated 282 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 05:26:12.029660 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa67a44-a54f-4e4e-a578-d6ce045aa1be admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:26:12.040373 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa67a44-a54f-4e4e-a578-d6ce045aa1be admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:26:12.044966 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9a351-1be5-4167-bd29-f301587b1500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:26:12.045572 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9a351-1be5-4167-bd29-f301587b1500 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:26:12.045858 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9a351-1be5-4167-bd29-f301587b1500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:26:12.046136 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9a351-1be5-4167-bd29-f301587b1500 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:26:12.046915 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9a351-1be5-4167-bd29-f301587b1500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:26:12.048519 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9a351-1be5-4167-bd29-f301587b1500 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:26:12.059325 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1440/2887] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:26:12.063185 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e2a29-7186-4236-8fe5-cd3fc4aa7f8a admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:26:12.073606 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e2a29-7186-4236-8fe5-cd3fc4aa7f8a admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:26:12.078159 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483e601-2674-469c-912d-68cdd796462a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:26:12.078758 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483e601-2674-469c-912d-68cdd796462a admin admin] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:26:12.079089 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483e601-2674-469c-912d-68cdd796462a admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:26:12.079363 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483e601-2674-469c-912d-68cdd796462a admin admin] PATH_INFO: `/v3/roles` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:26:12.080069 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5483e601-2674-469c-912d-68cdd796462a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:26:12.081661 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5483e601-2674-469c-912d-68cdd796462a admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:26:12.092199 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1448/2888] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Aug 3 05:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 05:26:12.096211 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9a351-1be5-4167-bd29-f301587b1500 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:26:12.106567 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9a351-1be5-4167-bd29-f301587b1500 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:26:12.111363 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8acd87-bbb3-484a-85f4-f32df79cdc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:26:12.112018 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8acd87-bbb3-484a-85f4-f32df79cdc43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:26:12.112303 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8acd87-bbb3-484a-85f4-f32df79cdc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:26:12.112582 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8acd87-bbb3-484a-85f4-f32df79cdc43 admin admin] PATH_INFO: `/v3/projects/ccee160dbb034ffd935a4ec135cee7e9/users/c0c737a8d33a4bc48002c12be91752bc/roles/b0e0084bb31348a582616b1421f3b10c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:26:12.128647 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8acd87-bbb3-484a-85f4-f32df79cdc43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccee160dbb034ffd935a4ec135cee7e9, user_id=c0c737a8d33a4bc48002c12be91752bc, role_id=b0e0084bb31348a582616b1421f3b10c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:26:12.131351 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8acd87-bbb3-484a-85f4-f32df79cdc43 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:26:12.225424 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1441/2889] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Aug 3 05:26:12 2022] PUT /identity/v3/projects/ccee160dbb034ffd935a4ec135cee7e9/users/c0c737a8d33a4bc48002c12be91752bc/roles/b0e0084bb31348a582616b1421f3b10c => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:26:12.238009 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ca5bf-0294-457a-a3d5-ae9d8451285f None None] REQUEST_METHOD: `POST` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:26:12.238302 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ca5bf-0294-457a-a3d5-ae9d8451285f None None] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:26:12.238610 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ca5bf-0294-457a-a3d5-ae9d8451285f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:26:12.245916 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:26:12.267317 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.auth.core [None req-f72ca5bf-0294-457a-a3d5-ae9d8451285f None None] MFA Rules not processed for user `c0c737a8d33a4bc48002c12be91752bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80356) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:26:12.268540 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-f72ca5bf-0294-457a-a3d5-ae9d8451285f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:26:12.333547 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1449/2890] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:26:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:26:12.345608 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8acd87-bbb3-484a-85f4-f32df79cdc43 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:26:12.347370 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6e8acd87-bbb3-484a-85f4-f32df79cdc43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:26:12.379167 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:26:12.418814 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8acd87-bbb3-484a-85f4-f32df79cdc43 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:26:12.422566 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf602e3c-0cc5-4207-9f1f-4963eacff7ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ca55d52244bf1bfe4e829b6880c20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:26:12.423223 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf602e3c-0cc5-4207-9f1f-4963eacff7ea service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:26:12.423560 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf602e3c-0cc5-4207-9f1f-4963eacff7ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:26:12.423836 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf602e3c-0cc5-4207-9f1f-4963eacff7ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:26:12.433203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf602e3c-0cc5-4207-9f1f-4963eacff7ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:26:12.434649 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf602e3c-0cc5-4207-9f1f-4963eacff7ea service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:26:12.444434 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1442/2891] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:26:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:26:22.334791 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=31 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:26:22.470943 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=18 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:26:50.759845 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ca5bf-0294-457a-a3d5-ae9d8451285f None None] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:26:50.823533 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ca5bf-0294-457a-a3d5-ae9d8451285f None None] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:26:50.828455 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d016d-9653-42b4-8ed3-c7fb0b73b0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:26:50.829106 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d016d-9653-42b4-8ed3-c7fb0b73b0c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:26:50.829393 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d016d-9653-42b4-8ed3-c7fb0b73b0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:26:50.829677 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d016d-9653-42b4-8ed3-c7fb0b73b0c2 admin admin] PATH_INFO: `/v3/users/c0c737a8d33a4bc48002c12be91752bc` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:26:50.830864 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d016d-9653-42b4-8ed3-c7fb0b73b0c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0c737a8d33a4bc48002c12be91752bc)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:26:50.833444 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d016d-9653-42b4-8ed3-c7fb0b73b0c2 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:26:51.031923 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-bb1d016d-9653-42b4-8ed3-c7fb0b73b0c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0c737a8d33a4bc48002c12be91752bc'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:51.074851 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-bb1d016d-9653-42b4-8ed3-c7fb0b73b0c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0c737a8d33a4bc48002c12be91752bc'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:51.075674 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-bb1d016d-9653-42b4-8ed3-c7fb0b73b0c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0c737a8d33a4bc48002c12be91752bc'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:51.089993 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-bb1d016d-9653-42b4-8ed3-c7fb0b73b0c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0c737a8d33a4bc48002c12be91752bc'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:51.090819 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-bb1d016d-9653-42b4-8ed3-c7fb0b73b0c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0c737a8d33a4bc48002c12be91752bc'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:51.103739 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1450/2892] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:26:50 2022] DELETE /identity/v3/users/c0c737a8d33a4bc48002c12be91752bc => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:26:51.108567 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9a397-dc98-4076-a213-efa0e0faf3c3 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:26:51.109065 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9a397-dc98-4076-a213-efa0e0faf3c3 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:26:51.109174 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9a397-dc98-4076-a213-efa0e0faf3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:26:51.128393 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-e6b9a397-dc98-4076-a213-efa0e0faf3c3 None None] MFA Rules not processed for user `d53b9368b2734205a2bdcd0bc2796161`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 05:26:51.130165 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e6b9a397-dc98-4076-a213-efa0e0faf3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:26:51.189824 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1443/2893] 10.222.0.40 () {58 vars in 958 bytes} [Wed Aug 3 05:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 05:26:51.200017 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d016d-9653-42b4-8ed3-c7fb0b73b0c2 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:26:51.201427 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-bb1d016d-9653-42b4-8ed3-c7fb0b73b0c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:26:51.259203 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d016d-9653-42b4-8ed3-c7fb0b73b0c2 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:26:51.263097 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ca7bd-5a88-43db-8b36-4643bd391d10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:26:51.263728 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ca7bd-5a88-43db-8b36-4643bd391d10 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:26:51.264015 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ca7bd-5a88-43db-8b36-4643bd391d10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:26:51.264294 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ca7bd-5a88-43db-8b36-4643bd391d10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:26:51.273611 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ca7bd-5a88-43db-8b36-4643bd391d10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:26:51.275072 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ca7bd-5a88-43db-8b36-4643bd391d10 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:26:51.284526 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1451/2894] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:26:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:26:51.636756 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9a397-dc98-4076-a213-efa0e0faf3c3 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:26:51.641837 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e6b9a397-dc98-4076-a213-efa0e0faf3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:26:51.650487 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9a397-dc98-4076-a213-efa0e0faf3c3 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:26:51.655420 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd198d-cd9a-4d50-b333-65cfb5489469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:26:51.656075 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd198d-cd9a-4d50-b333-65cfb5489469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:26:51.656372 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd198d-cd9a-4d50-b333-65cfb5489469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:26:51.656649 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd198d-cd9a-4d50-b333-65cfb5489469 admin admin] PATH_INFO: `/v3/projects/ccee160dbb034ffd935a4ec135cee7e9` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:26:51.657873 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd198d-cd9a-4d50-b333-65cfb5489469 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccee160dbb034ffd935a4ec135cee7e9)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:26:51.659822 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd198d-cd9a-4d50-b333-65cfb5489469 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:26:51.749130 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8ccd198d-cd9a-4d50-b333-65cfb5489469 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccee160dbb034ffd935a4ec135cee7e9', 'request_id': 'req-8ccd198d-cd9a-4d50-b333-65cfb5489469'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:51.789181 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8ccd198d-cd9a-4d50-b333-65cfb5489469 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccee160dbb034ffd935a4ec135cee7e9', 'request_id': 'req-8ccd198d-cd9a-4d50-b333-65cfb5489469'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:51.795192 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8ccd198d-cd9a-4d50-b333-65cfb5489469 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccee160dbb034ffd935a4ec135cee7e9', 'request_id': 'req-8ccd198d-cd9a-4d50-b333-65cfb5489469'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:51.807245 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8ccd198d-cd9a-4d50-b333-65cfb5489469 admin admin] The token cache is being invalidate because project ccee160dbb034ffd935a4ec135cee7e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:26:51.807556 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8ccd198d-cd9a-4d50-b333-65cfb5489469 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:51.808203 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8ccd198d-cd9a-4d50-b333-65cfb5489469 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:26:51.809709 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1444/2895] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:26:51 2022] DELETE /identity/v3/projects/ccee160dbb034ffd935a4ec135cee7e9 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:27:01.280246 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:27:01.804116 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:27:42.989747 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ca7bd-5a88-43db-8b36-4643bd391d10 service neutron] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:27:42.999585 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ca7bd-5a88-43db-8b36-4643bd391d10 service neutron] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:27:43.049874 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57fe7e9-12c9-4f33-839b-93722adfc3a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff15ea4cbc64b068aa5f6586b5bff87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:27:43.050491 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57fe7e9-12c9-4f33-839b-93722adfc3a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:27:43.050774 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57fe7e9-12c9-4f33-839b-93722adfc3a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:27:43.051080 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57fe7e9-12c9-4f33-839b-93722adfc3a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:27:43.052859 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-d57fe7e9-12c9-4f33-839b-93722adfc3a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:27:43.116921 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57fe7e9-12c9-4f33-839b-93722adfc3a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:27:43.119036 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57fe7e9-12c9-4f33-839b-93722adfc3a7 service neutron] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:27:43.128998 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1452/2896] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:27:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:27:46.303106 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd198d-cd9a-4d50-b333-65cfb5489469 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:27:46.304616 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8ccd198d-cd9a-4d50-b333-65cfb5489469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:27:46.376953 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd198d-cd9a-4d50-b333-65cfb5489469 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:27:46.381905 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec0e1c6-4778-45ad-8eb9-6b0520cea113 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59846d88f4314c2382aa5cb8bae5053b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:27:46.382525 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0e1c6-4778-45ad-8eb9-6b0520cea113 service placement] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:27:46.382810 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0e1c6-4778-45ad-8eb9-6b0520cea113 service placement] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:27:46.383121 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0e1c6-4778-45ad-8eb9-6b0520cea113 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:27:46.385545 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7ec0e1c6-4778-45ad-8eb9-6b0520cea113 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:27:46.396835 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec0e1c6-4778-45ad-8eb9-6b0520cea113 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:27:46.398397 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec0e1c6-4778-45ad-8eb9-6b0520cea113 service placement] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:27:46.408380 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1445/2897] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:27:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:27:49.738799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57fe7e9-12c9-4f33-839b-93722adfc3a7 service neutron] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:27:49.751161 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57fe7e9-12c9-4f33-839b-93722adfc3a7 service neutron] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:27:49.757163 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5b4c2-f7d3-44b3-92f9-eb0ffced287b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59846d88f4314c2382aa5cb8bae5053b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:27:49.757988 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5b4c2-f7d3-44b3-92f9-eb0ffced287b service placement] REQUEST_METHOD: `GET` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:27:49.758575 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5b4c2-f7d3-44b3-92f9-eb0ffced287b service placement] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:27:49.759100 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5b4c2-f7d3-44b3-92f9-eb0ffced287b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:27:49.761726 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-94a5b4c2-f7d3-44b3-92f9-eb0ffced287b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:27:49.775757 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a5b4c2-f7d3-44b3-92f9-eb0ffced287b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:27:49.777374 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a5b4c2-f7d3-44b3-92f9-eb0ffced287b service placement] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:27:49.788107 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1453/2898] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:27:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 05:27:49.847587 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec0e1c6-4778-45ad-8eb9-6b0520cea113 service placement] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:27:49.849077 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7ec0e1c6-4778-45ad-8eb9-6b0520cea113 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:27:49.872337 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec0e1c6-4778-45ad-8eb9-6b0520cea113 service placement] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:27:49.873667 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45973f-5688-434d-9787-1bff402257bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:27:49.874314 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45973f-5688-434d-9787-1bff402257bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:27:49.874618 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45973f-5688-434d-9787-1bff402257bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:27:49.874923 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45973f-5688-434d-9787-1bff402257bf admin admin] PATH_INFO: `/v3/users/ac3954ab39014ef3819111a6fe8142b7` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:27:49.876113 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45973f-5688-434d-9787-1bff402257bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac3954ab39014ef3819111a6fe8142b7)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:27:49.878038 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45973f-5688-434d-9787-1bff402257bf admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:27:50.059427 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8f45973f-5688-434d-9787-1bff402257bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac3954ab39014ef3819111a6fe8142b7'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:27:50.124736 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8f45973f-5688-434d-9787-1bff402257bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac3954ab39014ef3819111a6fe8142b7'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:27:50.125277 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8f45973f-5688-434d-9787-1bff402257bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac3954ab39014ef3819111a6fe8142b7'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:27:50.139010 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8f45973f-5688-434d-9787-1bff402257bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac3954ab39014ef3819111a6fe8142b7'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:27:50.139680 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8f45973f-5688-434d-9787-1bff402257bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac3954ab39014ef3819111a6fe8142b7'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:27:50.151674 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1446/2899] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:27:49 2022] DELETE /identity/v3/users/ac3954ab39014ef3819111a6fe8142b7 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:27:50.422996 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5b4c2-f7d3-44b3-92f9-eb0ffced287b service placement] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:27:50.435786 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5b4c2-f7d3-44b3-92f9-eb0ffced287b service placement] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:27:50.497152 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2b7d9-564b-4940-a774-f35119186ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:27:50.497799 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2b7d9-564b-4940-a774-f35119186ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:27:50.498083 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2b7d9-564b-4940-a774-f35119186ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:27:50.498361 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2b7d9-564b-4940-a774-f35119186ec9 admin admin] PATH_INFO: `/v3/projects/3c986ef0f5ea46f9b5e788aaca042b06` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:27:50.499536 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f2b7d9-564b-4940-a774-f35119186ec9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c986ef0f5ea46f9b5e788aaca042b06)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:27:50.501465 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f2b7d9-564b-4940-a774-f35119186ec9 admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:27:50.627511 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:27:50.634292 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-12f2b7d9-564b-4940-a774-f35119186ec9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c986ef0f5ea46f9b5e788aaca042b06', 'request_id': 'req-12f2b7d9-564b-4940-a774-f35119186ec9'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:27:50.640542 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-12f2b7d9-564b-4940-a774-f35119186ec9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c986ef0f5ea46f9b5e788aaca042b06', 'request_id': 'req-12f2b7d9-564b-4940-a774-f35119186ec9'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:27:50.706077 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-12f2b7d9-564b-4940-a774-f35119186ec9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c986ef0f5ea46f9b5e788aaca042b06', 'request_id': 'req-12f2b7d9-564b-4940-a774-f35119186ec9'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:27:50.717565 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-12f2b7d9-564b-4940-a774-f35119186ec9 admin admin] The token cache is being invalidate because project 3c986ef0f5ea46f9b5e788aaca042b06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:27:50.718123 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-12f2b7d9-564b-4940-a774-f35119186ec9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:27:50.719211 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-12f2b7d9-564b-4940-a774-f35119186ec9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:27:50.720748 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1454/2900] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:27:50 2022] DELETE /identity/v3/projects/3c986ef0f5ea46f9b5e788aaca042b06 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:27:50.724792 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45973f-5688-434d-9787-1bff402257bf admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:27:50.726204 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8f45973f-5688-434d-9787-1bff402257bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:27:50.740754 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45973f-5688-434d-9787-1bff402257bf admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:27:50.798724 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd21bded-b0db-475a-910b-f3f6533372c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:27:50.799400 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd21bded-b0db-475a-910b-f3f6533372c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:27:50.799686 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd21bded-b0db-475a-910b-f3f6533372c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:27:50.799964 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd21bded-b0db-475a-910b-f3f6533372c7 admin admin] PATH_INFO: `/v3/users/f2d6f433ebaa40d2bc5dfaba438b676f` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:27:50.801134 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd21bded-b0db-475a-910b-f3f6533372c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2d6f433ebaa40d2bc5dfaba438b676f)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:27:50.803273 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd21bded-b0db-475a-910b-f3f6533372c7 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:27:50.808084 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:27:51.000456 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-fd21bded-b0db-475a-910b-f3f6533372c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2d6f433ebaa40d2bc5dfaba438b676f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:27:51.066861 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-fd21bded-b0db-475a-910b-f3f6533372c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2d6f433ebaa40d2bc5dfaba438b676f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:27:51.067560 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-fd21bded-b0db-475a-910b-f3f6533372c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2d6f433ebaa40d2bc5dfaba438b676f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:27:51.080420 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-fd21bded-b0db-475a-910b-f3f6533372c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2d6f433ebaa40d2bc5dfaba438b676f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:27:51.081066 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-fd21bded-b0db-475a-910b-f3f6533372c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2d6f433ebaa40d2bc5dfaba438b676f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:27:51.093763 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1447/2901] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Aug 3 05:27:50 2022] DELETE /identity/v3/users/f2d6f433ebaa40d2bc5dfaba438b676f => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:27:51.606031 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2b7d9-564b-4940-a774-f35119186ec9 admin admin] Authenticating user token {{(pid=80356) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:27:51.607528 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.fernet_utils [None req-12f2b7d9-564b-4940-a774-f35119186ec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80356) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:27:51.621692 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2b7d9-564b-4940-a774-f35119186ec9 admin admin] Validating token access rules against request {{(pid=80356) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:27:51.683114 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574ccb91-90a1-47e7-aae6-ee8a626e12ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd53b9368b2734205a2bdcd0bc2796161', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6917683e025f412781fd852422d27ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80356) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:27:51.687195 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ccb91-90a1-47e7-aae6-ee8a626e12ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:27:51.687195 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ccb91-90a1-47e7-aae6-ee8a626e12ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:27:51.687195 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ccb91-90a1-47e7-aae6-ee8a626e12ca admin admin] PATH_INFO: `/v3/projects/f3d47ea13c2d4e9181faf641275f034f` {{(pid=80356) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:27:51.687195 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574ccb91-90a1-47e7-aae6-ee8a626e12ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3d47ea13c2d4e9181faf641275f034f)` {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:27:51.687699 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574ccb91-90a1-47e7-aae6-ee8a626e12ca admin admin] RBAC: Authorization granted {{(pid=80356) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:27:51.839111 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-574ccb91-90a1-47e7-aae6-ee8a626e12ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3d47ea13c2d4e9181faf641275f034f', 'request_id': 'req-574ccb91-90a1-47e7-aae6-ee8a626e12ca'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:27:51.845328 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-574ccb91-90a1-47e7-aae6-ee8a626e12ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3d47ea13c2d4e9181faf641275f034f', 'request_id': 'req-574ccb91-90a1-47e7-aae6-ee8a626e12ca'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:27:51.911648 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-574ccb91-90a1-47e7-aae6-ee8a626e12ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3d47ea13c2d4e9181faf641275f034f', 'request_id': 'req-574ccb91-90a1-47e7-aae6-ee8a626e12ca'} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:27:51.922735 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-574ccb91-90a1-47e7-aae6-ee8a626e12ca admin admin] The token cache is being invalidate because project f3d47ea13c2d4e9181faf641275f034f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80356) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 05:27:51.923051 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-574ccb91-90a1-47e7-aae6-ee8a626e12ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:27:51.923682 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG keystone.notifications [None req-574ccb91-90a1-47e7-aae6-ee8a626e12ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80356) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 05:27:51.925187 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: [pid: 80356|app: 0|req: 1455/2902] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Aug 3 05:27:51 2022] DELETE /identity/v3/projects/f3d47ea13c2d4e9181faf641275f034f => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 05:28:01.090277 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=25,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:28:01.919630 dv-cinder-851279-4-iscsi devstack@keystone.service[80356]: DEBUG dbcounter [-] [80356] Writing DB stats keystone:SELECT=61,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80356) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 05:31:07.201832 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd21bded-b0db-475a-910b-f3f6533372c7 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 05:31:07.203269 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-fd21bded-b0db-475a-910b-f3f6533372c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:31:07.344677 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd21bded-b0db-475a-910b-f3f6533372c7 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 05:31:07.405776 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5bc62-774d-49ce-b8e1-e04441a66d2d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59846d88f4314c2382aa5cb8bae5053b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3763a37383456184a9ca9351b4ce03', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 05:31:07.406392 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5bc62-774d-49ce-b8e1-e04441a66d2d service placement] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 05:31:07.406673 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5bc62-774d-49ce-b8e1-e04441a66d2d service placement] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 05:31:07.406981 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5bc62-774d-49ce-b8e1-e04441a66d2d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 05:31:07.408736 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-abc5bc62-774d-49ce-b8e1-e04441a66d2d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 05:31:07.420419 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5bc62-774d-49ce-b8e1-e04441a66d2d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 05:31:07.421982 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5bc62-774d-49ce-b8e1-e04441a66d2d service placement] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 05:31:07.432728 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1448/2903] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Aug 3 05:31:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 05:31:17.427869 dv-cinder-851279-4-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}